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I. INTRODUCTION 


In the recent past, there possibly was a time when protection of vital electronic 
information was not considered a necessity and therefore not deemed to be a topic 
of common interest. Such a time is forever behind us. In our time, information is 
most often passed across a public telecommunication medium. Whether this medium 
be a telephone line or satellite link, there exist eavesdropping methods which are so 
sophisticated and efficient that no information is physically secure. How then is one 
to revert to the inherent privacy of the past? The answer to this question and thus 
the solution to concealment of information lie in the complex science of cryptography. 

Cryptography is the field involving the preparation of messages intended to be 
incomprehensible to all except those who legitimately possess the means to recover the 
original information [Ref 1]. At present, the fastest and most popular cryptosystems 
employ some convention of mapping a set of numbers representing data to another 
set of numbers (encryption). The recovery of data is done by simply reversing the 
mapping process so as to obtain the original content (decryption). Often, this type of 
mapping is governed by the notion of a key. In order to provide the essential element 
of secrecy, system users must provide this key which is normally a privately or semi- 
privately known string of characters or bits. For a cryptosystem to be completely 
secure, knowledge of both the mapping function and key is required to recover the 
original text from encrypted text. 

Of the cryptosystems which use the forementioned concept of a key, two distinct 
categories are made: secret-key and public-key. 

As suggested by the name, a cryptosystem is secret-key if the key must be 


secretly agreed upon prior to any parties being able to communicate through the 


system. In this arrangement, both parties normally have the same key which is used 
in both encryption and decryption. Algorithms implementing this scheme are labeled 
symmetric. Intuitively, one recognizes a severe restriction in the secret-key system: 
an advance agreement on the key over a secure channel. When such a channel is 
not readily available, the topic of this thesis, public-key cryptosystem (PKS), is the 
remedy. 

Most PKS systems use an asymmetric algorithm whereupon separate keys are 
required for encryption and decryption. This scheme allows the passing of keys, 
most likely encryption keys, over an unsecure channel without any compromise to 
the system’s safety. In boasting this versatile capability, however, public-key system 
must pay a price, namely a reduction in system speed [Ref 2]. Currently, PKS is much 
slower than secret-key, too slow for large quantities of data. For this reason, its use 
is often limited to the exchange of keys in secret-key systems. In the future, along 
with advancements in technology, perhaps this speed barrier will be lifted yielding 
more opportunity for the employment of PKS. 

It is in the spirit of this future that this thesis is presented. It is an in-depth 
study of the public-key cryptosystem. First, the mathematical basis behind PKS is 
covered so as to establish an essential background knowledge in a somewhat esoteric 
subject. Second, the capability of VLSI implementation of PKS is explored via a 
fast modulo exponentiator, a hardware device required in two of the most popular 
public-key systems. A vital component of the fast modulo exponentiator, a modulo 
reduction unit, is designed with MAGIC tools [Ref 3], validated with RNL simulation 
[Ref 4], and examined for possible use. Finally, to conclude the scope of this research, 
a completely novel approach to PKS is proposed: a possible implementation of neural 


networks in public-key cryptography. 


Il. MATHEMATICAL BASIS FOR THE 
DEVELOPMENT OF PUBLIC-KEY 
CRYPTOSYSTEMS 


Compared to the complexity of conventional engineering mathematics, the con- 
cepts behind the algorithms for public-key cryptosystem are elementary in nature yet 
without complete understanding of them, no initial familiarization to the system is 
possible. Due to this realization, this chapter concentrates heavily on the mathemat- 
ics of asymmetric cryptography. It provides a basic overview of modulo arithmetic, 
fast exponentiation, and discrete logarithm. It also outlines a background knowledge 
in artificial neural networks, a branch of engineering upon which a completely new 
angle in cryptography is based. Furthermore, the fundamentals of public-key cryp- 
tosystems are covered using two well-established examples, the Diffie-Hellman and 
RSA systems. Finally, the chapter concludes with the problem of cryptoanalysis: the 


purpose of all cryptosystems. 
A. MODULO ARITHMETIC 


Modulo arithmetic is a branch of integer mathematic best explained by an ex- 
ample. 


Simply, 


21 = 3(mod9) 


or 
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This operation is commonly described as 21 divided by 9 equals 2 with remainder 
of 3. 
When written as z = y(mod z), by convention z is said to be “congruent to y 


modulo z.” Congruency applies if and only if 
r=yt+kxz 


where k is any integer. Also y is called a residue mod z of z if and only if z = 
y(mod z). 

Note that —15(mod 6) = —3(mod 6). 

Clearly, for any z, y belongs to a complete set of residues {0,1,2...,z-—1}. From 
this complete set of residues, there exists a subset called a reduced set of residues 
which has elements relatively prime to the modulus z. For example, a complete set 
of residues modulo 12 is {0, 1, 2,3, 4,5,6, 7,8,9, 10,11}. From this, only {1,5, 7,11} 
does not have a common factor with 12 (0 excluded); it is therefore a reduced set 
(Ref 2]. : 

For a modulo prime, clearly the reduced set of residues contains all elements of 
the complete set except for 0. Therefore for a prime n, the reduced set of residues 
has (n — 1) elements. In addition, generally the reduced set of residues for a product 
of two primes m and n has ((m —1)(n — 1)) elements and that for a prime power n’ 
has (n — 1)n'"-)) elements. Commonly, the number of elements in a reduced set of 
residues for modulo n is referred to as the Euler Totient function ¢(n) [Ref 2]. Table 
2.1 shows ¢(n) for several n [Ref 2]. 

Like normal integer arithmetic, addition and multiplication in integer modulo n 


abide by the laws of associativity, commutativity and distributivity [Ref 2]. 


Theorem 1 (Ref 2): 


(a + 6)(mod n) = (a mod n+ 6 mod n) mod n 
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Theorem 2 [Ref 2]: 


ab(mod n) = (a mod n x b mod n) mod n 


These two theorems form the basis for the development of fast modulo expo- 


nentiation. 
B. FAST MODULO EXPONENTIATION 


Many public-key cryptosystem requires the computation of z* mod n, with n 
and & being extremely large numbers (in excess of 256 bits.) A naive solution would 
be to multiply by z a repetition of k — 1 times then taking the modulo of the large 
result. At best, this is both cumbersome and inefficient for today’s computers due 


to finite word length limit. Fortunately, there is an algorithm which avoids this 
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TABLE 2.2: EXAMPLE FAST EXPONENTIATION FOR 5!° 


straightforward method: fast modular exponentiation [Ref 5]. 

Taking advantage of Theorem 2, the exponentiation is faster when performed 
by repeated squaring operations coupled with conditional multiplication by the par- 
tial product according to the binary representation of the exponent. This is best 


explained by an example. 


Example: 
Suppose we are required to find 5!° mod 9. 
let.2 = 3k = 10 9 


Using ppp = 1 and 


Q¢-1 e 
z x io f k; =i 
pp; = PPi— Vga) 3 
PPi-1 if k; =0 


k in binary is 1010. In accordance to k, bit by bit from least significant bit 
(LSB) first, the squaring of x occurs iteratively for every k bit (0 or 1) but the result 
is multiplied by the partial product only when k bit is 1. All the while, modulo 
operation is performed in each squaring or multiplication in order to maintain a 
manageable intermediate result. The partial product is always initialized to 1 (partial 
product at iteration step 0, ppp = 1). Let’s examine Table 2.2 for clarity. From the 
result of Table 2.2, indeed we have accomplished 5!°. O 

If we incorporate the modulo operation into each iteration according to Theorem 


2, the modulo problem is also solved. Table 2.3 incorporates modulo reduction to 


Rin [Squce pe [Mules 
1 (5°) mod 9 = 5 
(57) mod 9 = 7x 
(77) mod 9 = 4 

(47) mod 9 = 7x | 7 mod 9 














1 mod 9 





TABLE 2.3: EXAMPLE FAST EXPONENTIATION AND MODULO OF 5”° mod 9 
the previous example. 


Example: 
51° mod 9 

Table 2.3 outlines in detail the process until a partial product of 49 is obtained. 
Note that the result of the square operation becomes the number to be squared in the 
next iteration. Also the previous partial product is the number in the multiplying 
operation if the’k bit is 1. In this saan, since 49 mod 9 = 4, indeed 5!° mod 9 
(which also equals 4) is performed. O 

In this example the savings in multiplications is 4 (5 versus 9 using the naive 

method). For larger number applications, let a be the number of binary bits of the 
exponent k and 6 be log, a. Using fast exponentiation, the number of multiplications 
(call it X) is bounded by 6+1 < X < 2b +1 depending on the number of 1’s and 
0’s in k. X with fast exponentiation grows linearly in length of k and is considerably . 
smaller then X obtained by the straightforward method of multiplying by k —1 times 
(Ref 5]. | | 

Appendix A contains a C program implementing fast modular exponentiation 
using the above algorithm. It should be noted that the program is not suitable for 
numbers exceeding the capability of the computer. Most computers have 32 bits res- 
olution therefore results which are greater than 32 bits are likely to be too large. This 


limitation, however, is resolved by using hardware for fast modular exponentiation 


i 


as will be shown in Chapter III. 
C. DISCRETE LOGARITHM 


Discrete logarithm is the branch of mathematics centered on the solution to the 


exponent of a powered number; namely, finding z in a* = 6 mod n when given a, 6,n. 


Example: 


a= $3194 


3! mod 11 =3 
3* mod 11 =9 
J miodelie=s 
3* mod 11 = 4 


SO t= 4, 


Given a large modulus n and a, b (greater than 100 digits magnitude), discrete 
logarithm is classified as a non—deterministic polynomials problem; the solution to 
which is extremely difficult and impractical to derive [ Ref 6]. Therefore its use is 


prevalent throughout many public-key cryptosystems. 
D. INVERSES 


Unlike integer arithmetic, modulo arithmetic often has inverses. Given a € 


{0,n — 1}, there could be a unique 6 € {0,n — 1} such that 
ab(mod n) = 1 [Ref 2] 


A systematic method to compute inverses involves the notion of the greatest 


common divisor (gcd). Conventionally, gcd(a, 6) is an integer c such that a/c and 
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b/c result in the smallest possible integer value. For example, gcd(8,12) = 4 but 
gcd(8, 16) = 8. 
From the mathematics of gcd, we pose: 


Lemma 1 [Ref 2]: if ged(a,n) = 1 then 
a; mod n # a; mod n;0 <2,7 <n 
Fermat’s Theorem [Ref 2]: p is a prime and gcd(a,p) = 1 then 
a?-1)(mod p) =1 
Theorem 3 (Ref 2]:if gcd(a,n) = 1 then an a~',0 < a~' < n exists such that 


aa~' = 1(mod n) 


Theorem 4 [Ref 2]: if gcd(a,n) = 1 then 


a®™) mod n= 1 


Recall ¢(n) is the number of elements in a reduced set of residues (Table 2.1). 

From the above Theorems, Euclid’s algorithm is developed to find gcd(a,n) as 
well as inverse a~'(mod n) of a mod n. It is not within the scope of this study to 
detail the foundation of this algorithm. If further information is preferred, reference 
2 is suggested for consultation. For the purpose of this thesis, C programs for gcd 


and inverse are provided in Appendix A [Ref 2]. 
E. ARTIFICIAL NEURAL NETWORK 


In 1985, Ackley, Hinton and Sejnowski [Ref 7] applied a back-propagation neural 
network to encode orthogonal binary vectors of length N using log2N hidden units. 


Following this, Cottrell, Munro and Zipser [Ref 8] used the same type of network to 


achieve image (data) compression. Both fers two application examples involved a 
special form of mapping via neural networks and, thus, suggested a possible use in 
cryptography. In fact, they are inspirational for the work of Chapter IV in this thesis 
which explores in detail the possibility of implementing neural networks in a novel 
public-key cryptosystem. In light of this, this section provides a basic understanding 
of neural networks, especially the back—propagation neural network. 

A formal definition of a neural network is: 

”A neural network is a parallel, distributed information processing structure con- 
sisting of processing elements (which can possess a local memory and can carry out 
localized information processing operations) interconnected via unidirectional signal 
channels called connections. Each processing element has a single output connection 
that branches into as many collateral connections as desired; each carries the same 
signal— the processing element ouput. This ouput signal can be of any mathematical 
types. The information of each element can be ranoiga with the restriction that it 
must be completely local; it must depend only on the current values of arriving input 
signals at and on values in local memory.” [Ref 9] 

Having defined a neural network, the basic unit, a processing element, is shown 
in Figure 2.1. The processing element has many input connections combined by a 
simple summation. The combination is then transformed through a transfer function. 
The function of interest here is a hyperbolic tangent. The single ouput of the element 
is fanned out to several ouput paths which then become inputs of other elements. The 
ouput to input connections each has a corresponding weight. Since the connections 
prior to entering the elements are modified by the weights, the summation within 
each element is a weighted sum. The actual mathematical process within an element 


is thus: 


AOD wijti); t = layer; 7 = number of weights 
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Figure 2.1: A Processing Element 


An overall neural network consists of many processing elements joined together 
as previously discussed. A typical neural network, a back-propagation network in 
this case, is shown in Figure 2.2 [Ref 10]. For organization purpose, processing 
elements are grouped into layers. A normal network is composed of two layers with 
connections to the outside world: an input buffer where data is entered and an output 
buffer where the response of the network to the given input is stored. Layers between 
the input and ouput layers are named hidden layers [Ref 10]. 

There are currently many types of neural networks designed for multitude of 
applications. For the purpose of encoding and decoding in a cryptosystem where the 
mapping of input to output is almost always non-linear, a most suitable network is 
the back-propagation type. 

A back-propagation neural network is A 3 to 5 layer network that behaves as an 
interpolative—associative mapping scheme. That is it has the ability to learn map- 
ping by generalizing input/ouput pairs relationship [Ref 9]. Moreover, the network 
employs a supervised, delta-rule learning scheme whereupon the input stimulus and 
corresponding output are first presented to the system which in turn reduces the 


error between the actual output of each element and the desired ouput and gradually 


Il 
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Figure 2.2: A Back-Propagation Network (Ref 10] 





configures its weights to achieve the desired input/ouput mapping. After learning is 
accomplished, the error is reduced to minimum and the actual outputs of all inputs 
of interest will be approximately equaled to the theoretical output [Ref 10]. 

Having covered the necessary basics, the mathematical background for the back- 
propagation network is now provided. In order to establish a common convention, 


the notations used for this development is as follows. 


zi = current output of j** neuron in layer s, 


wit = connection weights joining z;, neuron in layer [s-1] to j** neuron in layer 


Ss, 


e I; ee = weight summation of inputs to j** neuron in layer s. 


The mathematical process for single back-propagation element is: 


a = AD (wh ay) = £(0") 
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Given that the network has some global error function E, the critical parameter 


that is fed back through the layers is defined as: 
el! = -9E/ar!" 


where el is the local error of processing element 7 in layer s. Furthermore, 


using the chain rule twice yields: 


ef = FE") Dek wi). 
k 


The main mechanism in the back-propagation network is to forward the input to 
the output, determine the error at the output, then propagate the errors back using 
the above equations.. Given knowledge of local errors, the final aim is to minimize 
the global error by modifying the weights. 

This is done by using the gradient rule which dictates that the weights change 


in the direction of minimum error. 


Awl!!! = —k(dE/dw'?!) 


where k is a learning coefficient. 
Again using the chain rule: 


OE/dw!? os (OE/ar'")\(ar'!!/dw!!!) -_ ell le-4 


Aw} = keryal, 


For an in-depth derivation of all forementioned equations, the reader is referred 


to references 9 and 10. 
Using the above equations in several iterations, an algorithm for the back- 


propagation network can be developed to train the network weights in converging to 
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a given set of training data: inputs and corresponding outputs. It is not within the 
scope of this research to derive or show the algorithm; however, such an algorithm 
can be found in reference 9. In Chapter IV, a specific software package, Neuralware, 
will be utilize to set up a back-propagation network. The network will train with 
specific mapping functions so as to accomplish an encryption and decryption scheme 
in a newly—proposed “pseudo” public-key cryptosystem. 

This concludes the necessary background in mathematic. We are now equipped 


with enough knowledge to explore the core of the public-key cryptosystem. 
F. THE PUBLIC-KEY CRYPTOSYSTEM 


The single foundation upon which all asymmetric cryptosystems are built is that 
of the one-way function. Such a function is practical to solve in one direction but 
within a range it is computationally infeasible for any algorithm to invert the solution 
taken over a range of elements [Ref 11]. A formal definition of a one-way function is 
beyond the scope of this study. An informal definition is that a one-way function is 
one in which for f : rz — y, it is easy to find y = f(z) given z. However, given y, it 
is difficult to compute z such that f(z) = y [Ref 12]. For use in cryptography, the 
difficulty must be great enough so as to render the solution impractical. 

Currently we have a few one-way functions which are utilized exclusively in the 
public-key system. A good example of a one-way function is integer multiplication. 
Whereas the multiplication of large integers is relatively easy with current technol- 
ogy, the factoring of a large integer is time-consuming to the point of infeasibility. 
Another important example is modular exponentiation with large exponents. As 
previously discussed, fast exponentiation techniques makes the exponentiation prac- 
tical. However, even with the best current algorithms and technology, the solution 


of a discrete logarithmic problem of such magnitude remains unattainable within a 
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reasonable time [Ref 13]. To see how the two suggested one-way functions are used 
in public-key cryptosystems, in-depth studies of two systems are now provided: the 


Diffie-Hellman and RSA cryptosystems. 


1. The Diffie-Hellman Scheme for Public-Key Cryp- 
tosystem 


The first system to achieve the notoriety of a true public-key system was 
proposed by Diffie and Hellman seminal paper in 1976 [Ref 14]. It is in this paper 
that the discrete logarithm problem was first proposed as a candidate for a one-way 
function. The scheme is best summarized as follows. 

Let n be a large integer and g, another integer, such that g € {1,n — 1}. 
Parties A and B establish n and g over insecure channels. A then chooses a large 
integer x and computes g” mod n while B chooses y and computes g¥ mod n. Next, A 
and B exchanges their perspective computations again Bei insecure channels without 
divulging z and y. At this point A has g¥ and n (possibly compromised over unsecured 
channels) and z which was never communicated to anyone. Similarly, B has g*,n 
and y. A and B can construct the key as follows. 

for A: key = (g¥)* mod n 

for B: key = (g7)¥ mod n 


(g¥)? mod n = (g*)¥ mod n 


Clearly A and B now have the same key (g7)¥ mod n which can be used — 
for any er otceraphy systems. Because the operation of exponentiation with large 
exponent is slow, Diffie-Hellman is proposed only to make keys for faster private-key 
system such as DES so that the key will not be compromised [Ref 12 ]. 

Even if a cryptanalyst was able to intercept the exchanges for g,n, g7 mod n 


and g¥ mod n, he faces the problem of finding z and y from his known data. He must 
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Figure 2.3: Block Diagram of Diffie-Hellman Cryptosystem 
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solve a discrete logarithm problem, an NP class problem, which, to date, is accepted 
to be infeasible within certain time restraints [Ref 13]. A summarizing block diagram 
of the Diffie-Hellman cryptosystem is provided in Figure 2.3. Moreover, an example 


of its application is hereby offered. 


Example [Ref 13]: 


Let g =7 and n =2 x 739(7!*9 — 1)/6 + 1. 
Party A chooses a secret z, compute and send 7” to B. 


B receives 77= 


1274021801199739468824269244334322849749382042586931621654557735 290322 
914679095998681860978813046595166455458144280588076766033781 


Party B chooses a secret y, compute and send 7” to A. 


A receives 7¥= 


1801622852874531024447828348348367 99895015967046695346697 313025121734 


05995377 2058475958117691062538069210165184866 2362137934026803049 


Now both A and B can compute 77% and mod it with n to establish secret 
key 77¥ mod n. Since a party other than A and B does not know either z or y in this 
case, it is infeasible to attempt finding. 77%. 

Note: The numbers in this example are obtained from reference 13 where 
neither z nor y was divulged. This author has been unable to find their values. In 
the original article, a challenge of 100 dollars was offered to anyone who could solve 
for z and y and thus 779.0 

Presently, the Diffie-Hellman scheme remains trustworthy because the dis- 


crete logarithm problem is still a difficult one to solve. Nevertheless, no one has 
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proven beyond a doubt that it is impossible to solve. In fact, many algorithms do 
exist which can derive the solution. The only setback is that even the best of them 
is not fast enough with current technology. For more safety, the integers z and y can 
simply be increased in magnitude and for the worst case, an establishment of new 


key within an acceptable time interval can render any cryptoanalysis harmless. 
2. The RSA Cryptosystem 


Invented in 1978, the Rivest, Shamir and Adleman (RSA) public-key cryp- 
tosystem incorporates two one-way functions: the discrete ees and factoriza- 
tion problems. The security guaranteed by this system is so sound that since its in- 
ception until present, it has been accepted as the most popular method of public-key 
encryption [Ref 15]. The elegance and subtle power of the RSA system is summarized 
as follows. 

Party A generates 2 random primes of approximately 130 bits each, p and 
q. The product pq is then computed and called n. The number of reduced residues 
elements is next obtained: ¢(n) = (p — 1)(q — 1) (see Table 2.1). In turn, an integer 
e is generated such that gcd(e, ¢(n)) = 1. A now has the public key < e,n > which 
can be published to B through insecured channels. 

Having the public key, party B can encrypt a message by transforming the 


message into an integer value m. m is then encrypt by: 
Encryp(m) = m* mod n 


In order to be able to decipher Encryp(m), A must make a private key from 


o(n) and e. Such a key, D, is found by using Euclid’s algorithm (Appendix A) so 
that, 


De = 1 mod ¢(n) 
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Decrypt with Private Kay <D.n> 
decrypt = Encrypt{im) - D mod a 





Figure 2.4: Block Diagram of RSA Cryptosystem 


Once D is found, the deciphering is simply done by, 
Deciph(Encryp(m)) = (Encryp(m))? mod n 


Proof [Ref 6]: 


Given all parameters above, by Euler’s Theorem: 


if De = 1 mod (¢) + m?* =mmodn 


—+ m?’* modn =m 


Figure 2.4 clarifies the process. In addition, a pedagogical example of RSA 


at work is shown below. 
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Example: 

(Use actual Appendix A programs ) 
Let p=73q=13 3 n=7 x 13 = 91; O(n) = (7 — 1)(138 — 1) = 72 
Pick e = 5 and D = 29 such that De = 2¢(n) +1 = 145 


Message m = 23 
Encryp(m) = 23° mod 91 = 4. 


Decryp(m) = 4*° mod 91 = 23.0 


Judging solely on the above example, it might not seem obvious that the 
RSA system is safe. The reason is because the example’s numbers are small. As 
stated earlier, with p and q both being about 130 bits, their product,n, can range 
in excess of 160 bits. In turn, e and D are also large numbers. Given this kind of 
range, to crack the code, one must face the discrete logarithm as well as factorization. 
To date, the factorization of a large product of primes remains unsolvable within a 


feasible time [Ref 2]. This fact is further examined in the next section, cryptoanalysis. 
G. CRYPTOANALYSIS 


The art of breaking cryptographic code is called cryptoanalysis. Since there are 
many public-key systems, the cryptoanalysis of only the RSA system is discussed so 
as to provide a TGR ol how difficult it is and thereby prove its soundness. 

The gist behind breaking the RSA system is the ability to solve for both the 
discrete logarithm and factorization problems. The latter of the two is the most 
difficult so the discrete logarithm problem will be the first to be explored. 

Given the public key < e,n > and let’s assume we were somehow able to factor 
n and therefore know p and q. We can now use Euclid’s algorithm the same way as if 


the sender would to make his/her private key. Take the example in the RSA section. 
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Knowing p and q we can compute ¢(n) = (p— 1)(q—-—1) 
Use Euclid’s algorithm to find the secret key D such that 


De = 1 mod ¢(n) 
With D, the sender’s encryption can be intercepted and decrypted by 


encryp(m)? mod n 


We have done the easy part. So far we assumed to know the two prime factors 
of the modulo n in the public key < e,n >. The main insurance of the RSA system 
is the derivation of the two factors p and q [Ref 15]. Whereas the cryptographer 
only has to come up with two primes, a difficult task but not impossible with the 
primes being about 130 bits, the cryptoanalyst, in order to recover the two primes to 
compute ¢(n), must face the grim task of factoring a number in excess of 260 digits 
within a finite time limit. This leads to the topic of factorization which will also 
be exploited as the safety basis for the later proposed cryptosytem based on neural 


network. 
1. Factorization 


A factorization problem has no current classification but the consensus is 
that it is neither a Polynomial (P) nor Nondeterministic Polynomial (NP)- Complete 
problem [Ref 16]. It is loosely described as a Nondeterministic Polynomial Indistin- 
guishable (NPI) problem [ Ref 16]. An algorithm is said to run in polynomial time 


(P) if there are constants A and c such that the running time for all inputs of length 


rs | 


k is Ak® for all &. All P problems are deterministic and P-time bounded. An al- 
gorithm is deterministic if at each step of the computation, the next step is unique. 
P-time bounded means that the execution is in polynomial time since its complexity 
is bounded by a polynomial in the input length. An algorithm is said to run in 
NP time if there are no known deterministic P-time solution. In NP problems, at 
each step of computation, decision problems on the next step exist. To systemati- 
cally solve an NP problem requires exponential time. A subset of NP problems, an 
NP-complete problem surfaces when P=NP. NP-complete problems are considered 
as the most difficult class in NP. An NPI problem is basically defined as having the 
level of difficulty in between NP and NP-complete. Factorization, an NPI problem, 
can not be solved in P-time and is not a member of NP-complete [Ref 2]. 

In order to be convinced that factorization of large numbers is at this time 
insurmountable, we examine the most straightforward and therefore easiest method. 
Given a number n to be factorized, we compute ,/n and round it to the next integer 
value, m. We then use m as the final index of a for to loop beginning with 1. In each 
iteration of the loop, the operation (n mod index) is performed until the result is 0 
notifying that an integer factor is found. Considering the speed of the computer, this 
is not a bad method of factorization if n is within a certain range of digits in length. 
However, this limit is what is exploited in public-key system (n is more than 130 
digits in length.) The shortcoming of this method is explored using Matlab program 
on an IBM ’486, 50 MHz, 16 MBytes (Appendix A). The result is shown in Table 
2.4. | 

Undisputably, with n being at least 100 decimal bits in the RSA system, the 
method above, although possible, is hardly feasible if exhaustive search is required. 

Fortunately, the mathematics of factoring have long surpassed the simplicity 


of the forementioned method. Currently there are established algorithms as well as 
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— ene 


Digits factorized | Aprroximate time | 























10 less than lmsec 
15 1.5sec 

20 15min 

29 28hr 

30 She 

40 3000 centuries * 


* Estimate 


TABLE 2.4: EXHAUSTIVE FACTORIZATION WITH ONE ’486 COMPUTER 


on-going researches which could reduce the time factor at a phenomenal rate. 

As a result of a concerted effort initiated in 1982, the mathematics de- 
partment at Sandia National Laboratory established some tangible bounds on the 
computational feasibility of factoring large numbers. The outcome, using a Cray 
X-MP computer, was within a range of 7.2 minutes to 32 hours for numbers varying 
from 55 to 77 digits in length [Ref 17]. 

In a separate study by Ronald Rivest [Ref 15], it is proven that with the 
best algorithm available such as that of a quadratic sieve [Ref 18], a large prime 


composite integer can be factored with a running time proportional to: 


e1 /in(n)in(In(n)) 


In the range of interest(approximately 256 bits in length), for k bit number 


n, a crude approximation is: 


5x 199+(4/59) 


Using Sandia’s benchmark that a 75-digit number can be factored in about 
1 day [Ref 17] and the formula of Rivest’s article [Ref 15], Table 2.5 is derived [Ref 
17]. - 

Based on the data above, it is safe to surmise that the problem of factor- 


ization of large number will remain insurmountable for a long time given current 
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Number of digits | Number of operations | Solution time 


1 day 
259 days 





103 years 

9755 years 

70 thousand years 
36 million years 


TABLE 2.5: FACTORIZATION TIME WITH SANDIA’S BENCHMARK [REF 17] 


knowledge and technology. The exploitation of this problem in the RSA system and 


the neural network-based system of Chapter IV is hereby justified. 
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Il. HARDWARE DEVELOPMENT OF THE 
PUBLIC-KEY CRYPTOSYSTEM 


The feasibility of most popular public-key systems is heavily dependent upon 
the possibility of hardware sanlementaton Although the algorithm is theoretically 
simple, its software implementation is ee and highly limited to the resolution of 
the processor. Such problems are not worth tackling When, with the available VLSI 
technology, hardware implementation is faster and more efficient. 

The crux of many public-key a hardware rests on the ability to 
devise a fast exponentiation scheme where the exponent and modulus are extreme in 
length (greater than 256 bits). From our two sample cryptosystems, Diffie-Hellman 
and RSA, the fast exponentiation problem is essential in putting the theory to prac- 
tice. To familiarize the reader with the possibility for hardware implementation of 
existing public-key cryptosystems, this chaptel will develop in detail a hardware 


scheme for fast exponentiation based the recursive sum of residues algorithm. 


_A. MODULO EXPONENTIATION USING RECURSIVE 


SUM OF RESIDUES 


Currently the most popular working hardware for the RSA system performs 
exponentiation by repeated squaring Poatabions coupled with conditional multipli- 
cation. During each square or multiplication stage, modulo reduction is also incor- 
porated so as to maintain a small intermediate result [Ref 19]. The combination of 
squaring (considered as part of multiplication), multiplication and modulo reduction 
operations forms the core of fast exponentiation. Currently, there are two categories 


separating the various methods of implementations: 
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Serial 






Multiplier 





Figure 3.1: Block Diagram of over all exponentiation unit 


~ 


1. Multiplication and modulo reduction are done in tandem. As the partial prod- 
ucts are formed, a decision based on special algorithms is made on whether to 


perform a reduction on the product [Ref 19]. 


2. Multiplication and modulo reduction are done sequentially. The result of the 
multiplication is first obtained and then fed serially to the modulo reduction 


unit [Ref 19]. 


For the purpose of this thesis, only the fatten case (2) is considered. The under- 
lying reason behind this choice is simplicity which leads to a modular structure that 
in turn can easily be implemented in VLSI. Moreover, the first part of this hardware 
scheme, a serial multiplier, will not be delved into with details due to the abundance 
of such units already available. This leads us to focus on the hardware implementa- 
tion of the modulo reduction unit to which the eal of the serial multiplier is fed 


into in accordance to the basic block diagram of F igure 3.1 [Ref 19]. 
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1. Sum of Residues Reduction 


Our modulo reduction unit is based on the sum-—of-residues reduction 
method. That is the number, z, reduced by modulus, m, is expressed in the fol- 
lowing binary form: 


— > 2;2'"). z; = (0,1) 


|) 


The modulo reduction is 
n . 
z mod m = ()_2;2'"') mod m 
t=] 
Since modulo reduction is associative 
nm . 
z mod m = ()_ 2;(2'" mod m)) mod m 
t=] 
Summarizing, one performs the reduction as a conditional power of 2 re- 
duced by mod m (a residue) and a summation of all the resulting residues (hence 


sum of residues) [Ref 19]. 


Example: 
modulus m is 7, z = 10010 = 18 , 1 initialized to 1. 

Residues are at 2! and 2* due to positions of 1 in 10010. Respectively the 
residues are 2 mod 7 and 16 mod 7 which are 2 and 2. Hence Sor; =r} +r4 = 2+2 _ 
4, 


Table 3.1 summarizes the SOR process for the example which resulted in: 
(S"r;) mod 7 =4 mod 7 = 4 


Indeed 18 mod 7 = 4 
Given a modulus, residues can be obtained by a look-up table; however, 


this requires excessive space. Given n as the modulus length, a typical table size is n 
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shift x LSB First el residue 2*~! mod 7 


2° mod 7 = 1 






x 





















2! mod 7 = 2 = 
2? mod 7 = 4 = 0 
23 mod 7 = 1 ==. () 
2* mod 7 = 2 = 2 


. residues will repeat | >> resulting 
124124... residues = 4 
pattern 


TABLE 3.1: EXAMPLE SUM OF RESIDUES FOR 18 mod 7 









i= Ora or =m 
r, initialized to 1 
2xl—i ou | 2 x= 
2x2=7 <0 2 <2 —4 
2x4—-7>0/)2x4-T7=1 


2x l=7 < 02 T= 





TABLE 3.2: EXAMPLE RECURSIVE SOR FOR 18 mod 7 


by 2n. With n being greater than 256 bits, this would require extremely large data 
paths, undesirable in silicon implementation [Ref 19]. For this reason, it woul be 
more desirable to calculate the residues-as necessary A accordance with the given 
modulus. Fortunately, there is a simple recursive formula which allows for easy 
hardware calculation of residues: 


ith residues = r;; 2 = 2...n 


a 2ri-1 af f (2ry — e) 
Se ey (2r;_; — m > 0) 


r, initialized to 1 [Ref 19] 


Taking the previous example from Table 3.1 and incorporating into it the 
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Preduct of Serial Multitpher 


Resulting MSB Shifted Bit By Bit, LSB First 


(Sign Bit) 
Modulus in | 
2’s Complement a > ee 
x by # of bits (pou = Pi 
Modulus. = 


Multiply 
‘ 5 
r by Pi 
(Using left z 





shift registers ) 


r x Pi 


Result Output 


Figure 3.2: Modulo Reduction Unit 


recursive sum of residues method, the result of which is in Table 3.2, indeed the 
residues are the iterative pattern: 1,2,4,1,2,4,1... 


A diagram of an architecture using the sum of residues method for modulo 


‘reduction is provided in Figure 3.2 [Ref 19] . 


Respectively, M and R are two n-bit registers holding (—m), the two’s 
complement of the modulus, and r;, the current residue. Initially, the current residue 
is set to 1. As the system is clocked, the register is loaded with 2r; or 2r; — m, 
depending on the sign bit of the 2r; — m add. The accumulator sums those residues 
which are passed by the incoming bits of the serial multiplier’s product P. There’s 
an overhead amount of bits which must be taken into acount for the accumulator’s 
size. The necessary overhead bits are given in Figure 3.3 [Ref 19]. 

Having a sound understanding of the theory behind the ereniectiee in 
Figure 3.2, the next obstacle that must be cleared is the transformation of the theory 
to an actual VLSI layout. With some intuition and basic knowledge of logic circuit, a 


block diagram complete with logic units, inputs and outputs is developed and shown 
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Overhead (bits) 
On 


0 100 200 300 400 500 600 
Input Bit-Length 


Figure 3.3: Overhead Vs Input Bit 


in Figure 3.4. 

A few details in the transformation between Figures 3.2 and 3.4 are hereby 
stated for clarification. Whereas in Figure 3.2 a multiplier was used to obtain the 
correct residue for the accumulator, in the final design, a multiplexer is chosen to 
perform the multiplication. Also the left shift logical to obtain 2r; is finalized without 
a shift register but rather by hardwiring the outputs of the residues directly to the 
inputs of the first adden 

From a VLSI perspective of Figure 3.4, one sees that it is beneficial to devise 
a modular unit (shaded region) which could easily be assembled together to form a 
larger complete reduction unit satisfying the length of the modulus. To realize a 
single modular unit, only 2 master-slave flip flop’s (MSFF), 2 combinational adders 
and 2 2:1 multiplexers are pede The control for this unit alone and for the rest of 
the modular reduction device is a couple of simple two-phase clocks. The simplicity 
of this modular scheme is attractive. However, the cost is in silicon area and speed 


as we will see. 
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Figure 3.4: Block Diagram of 4-Bit SOR with Logic Units 
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Figure 3.5: MSFF Circuit Diagram 


B. VLSI LAYOUT DEVELOPMENT a 
1. Master Slave Flip Flop 


The desire for a simple control method, a two-phase clock, necessitates the 
use of a master-slave flip flop instead of a direct latch. In the first stage where 
the residues‘are computed, the adder uses the output of the flip flop (slave) while 
the output of the hardwired shift left or; is transferred to the input end of the flip 
flop(master). The same requirements for the flip flop are imposed in the accumulator 
unit where the flip flop must act as both the accumulator’s adder output register - 
(master) as well as accumulated input to the adder. 

The chosen circuit for our master-slave flip flop is shown in Figure 3.5 [Ref 
20]. 

Analysis of Figure 3.5 shows two cascading 2-phase static latch. This struc- 
ture is sound and efficient to implement. A minor problem of clock race is possible 
when clock is high and clockbar overlaps it causing a tendency for the input and feed- 
back signal to contest with the new value on the flip flop input [Ref 20]. Fortunately, 
for our purpose; this problem did not manifest itself as the feedback transistor is 


designed to “trickle”: transistor 8 is low [Ref 20]. The VLSI layout for the master- 
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Figure 3.6: MSFF Layout 


slave flip flop is given in Figure 3.6. It should be preempted that the design will 
be slightly alter later on in order to conform to the overall modularity of the entire 
modulo reduction unit. 

Silicon space for the MSFF is 64 x 135 zm?. SPICE analysis (Ref 21] on the 
layout determined a delay from input to output to be 10ns. The maximum speed 
of operation for the MSFF is 100Mhz. Since the input and output of the MSFF is 


inherent only to the single module, no effect from the other modules are of concern. 
2. Adder 


Due to the modularity of the design, the simplest approach is taken in the 
development of the two adders in the module. The chosen unit for both adders is 
a combinational adder with approximately equal sum and carry delays. Carries are 
allowed to ripple through the necessary modules. This choice is made mainly to 
conform to the modular structure. The ripple carry design does cost much in speed. 


The circuit diagram for the adder is shown in Figure 3.7 [Ref 20]. The appropriate 
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Figure 3.8: Adder Layout 
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layout follows in Figure 3.8. 

The adder layout sizes up to 73x 145 ym?. SPICE analysis Ref 21] of a single 
adder unit showed that the sum and carry delays are 4.8ns and 4.5ns respectively. 
From this result, intuition dictates that when the unit is put together for a larger 


modulus, the carrychain will be the limiting parameter for speed of operation. 
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Figure 3.9: MUX Function Block Circuit Diagram 


3. Multiplexer 


The reduction unit calls for the use of two 2:1 mux’s per bit of modulus. 
The first takes its select input from the sign bit of the sum of the first adder and 
output 2r; or 2r; — m as appropriate. The second simply acts as : multiplier with 
its select input as the single bit shifted in from the output of the serial multiplier 
and outputs the residues if the select is 1 and 0 if select is 0. In short it acts as a 
single bit multiplier. For our multiplexer, a function block design is used [Ref 22]. 
The circuit is shown in Figure 3.9 [Ref 22]. 

This is an NMOS device in which only one of the two inputs a, b is passed to 
the output depending on whether NMOS-1 or NMOS-2 is turned on. Only one NMOS 
gate can turn on at the time because the inputs to their gates are complements. 
Intuitively, the select input of the multiplexer is the input to the two gates. The 
VLSI layout is shown in Figure 3.10. 

Because of the simplicity of the circuit, the only delay is one transistor 
gate. Compared to the delay of the adder or flip flop, this is negligible and will not 
be delved into. The size of the layout is 32 x 33 um?. 
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Figure 3.10: Layout of MUX 


4. Modulo Reduction Unit 


Having all the necessary components, the entire modulo reduction unit can 
now be developed. As previously mentioned, a “modular” design is implemented 
in this thesis so that, depending on the size of the modulus, the entire unit can be 
constructed by simply cascading the same module together n times (modulus is n-bit 
in length.) Bearing this in mind, the layout for the module as well as a 4-bit modulus 
modulo reduction unit 1s shown in Figure 3.11. 

The foremost significance of the VLSI scheme for the modulo reduction unit 
is that it is simple in implementation and, above all, it works. Using a CFL program 
[Ref 3], the module can easily be generated into an n bit unit. Experimentally, RNL 
simulations were performed [Ref 3]. The results, which are enclosed in Appendix 
B, testify strongly on behalf of the unit’s functional capability. However, as to the 
efficiency in area and speed, the empirical data is debatable in support of different 
individual’s needs. 

Since the modulo reduction unit is designed mainly for modularity, the size 
of the entire structure grows geometrically with the number of bit that the unit is 
designed for. Each module per bit is sized at 73 x 672 um?. If n is the number of 


bits required to be modulo reduced, then n modules are needed. Disregarding the 
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Figure 3.11: Layout of 4-bit Modulo Reduction Unit 
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Figure 3.12: Size of Modulo Reduction Unit 
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Figure 3.13: Speed Performance of Modulo Reduction Unit From SPICE 


minimal effect of overhead bits (Figure 3.3), the size of a modulo reduction unit for 
n-bit modulus is n x 49056um?. Figure 3.12 is a plot relating the size of the unit to 
the number of bits. 

In regard to speed consideration, experimental data found the unit’s car- 
rychain to be the limiting factor. After SPICE simulation [Ref 21], Figure 3.13 was 


obtained to gauge the speed performance of the modulo reduction unit. 
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Since the carrychain imposes the speed limit in this design, intuitively, one 
can incorporate speed saving techniques such as various carry-look-ahead adders; 
however, this will alter the modularity structure. This is beyond the scope of the 
thesis but remains a viable avenue for speed improvement at the expense of silicon 
space. 

In summary, this chapter has provided the basic hardware building blocks 
for a fast exponentiation scheme with specific details on a modulo reduction unit. 
From this foundation, an RSA hardware implementation can easily be conceived. 
Such an implementation is necessary in many applications, one of which is the subject 
of the next chapter: a novel approach to PKS using neural networks. As will be 
explained in the following chapter, the hardware technology developed here will be a 


small integral part of a “pseudo” public-key cryptosystem based on neural networks. 
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IV. A NEURAL NETWORK-BASED 
PUBLIC-KEY CRYPTOSYSTEM 


Since all cryptosystems make use of some form of mapping functions to trans- 
form data to unintelligible code and then recover it, a neural network - inherently 
an excellent non-linear mapping technique — provides a viable choice for a medium 
from which a possible cryptosystem can be based upon. In examining this possibil- 
ity, this chapter presents an adaptation of the back-propagation neural network to 
a “pseudo” public-key arrangement. Strictly as an initial research, a simple require- 
ment of encrypting and decrypting a number representing any character or data is 
fulfilled via the network. Following examinations of the network, a key management 
system is then devised. As data are fed to the network in simulation of encrypting 
and decrypting, the problems and solutions to the system are discussed. Finally, 
a complete top-down block diagram of an entire cryptosystem based on the neural 


network of this study is proposed. 


A. EXPERIMENTS IMPLEMENTING A NEURAL NET- 
WORK IN CRYPTOSYSTEMS 


The neural network-based cryptosystem to be designed, a cipher system, re- 
quires two basic elements: a key management scheme and an algorithm for two-way 
mapping a set of numbers representing data. In this respect, it is fundamentally 
not far different than other cryptosystems. The differences surface only in the im- 
plementation of mapping. Whereas all existing system such as DES [Ref 23], once 
implemented in hardware, maps in a set pattern, a neural network can change its 


mapping any time by simply retraining its weights to new data. As it turns out, this 
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deviation from the norm is advantageous since it adds an extra level of protection. 
Namely, if the system is compromised, retraining and obtainment of new weights are 
neither a difficult nor time-consuming task [Ref 24, 25]. 

Before the network is presented, some background is in order. The system 
of this study is designed to map up to a set of 45 characters for encryption and 
decryption. Figure 4.1 is a block diagram of the system. From Figure 4.2 [Ref 26], 
the two networks for encryption and decryption are identical systems; they are both 
back—propagation networks composed of 4 inputs, 1 output, and three hidden layers 
of various sizes. 

Prior to proceeding with the explanations of Figure 4.1, it is stressed that this 
system is based mainly on the RSA system. As such, it simply takes a number, 
encrypts it to another number and decrypts it back. Like RSA, this is all the neural 
network is set up to do. For simplicity, this number represents a particular character; 
however, the relationship between the number and character is not eee in detail 
because this is a subject outside of the focus of this thesis. Furthermore, the input to 
the network of this research is only 16 bit in length. Again this is chosen for simplicity 
and clarity in an example system. It is not chosen for security. Like RSA in which 
system security rests on the key being numbers greater than 256 bit, the security of 
this system also depends upon the range of the input being greater than 256 bit. In 
fact, with the input being only 16 bit long, the system can be compromised within 
nanoseconds. However, successful cryptoanalysis of 256-bit inputs will be shown in 
Section 4.D.1 to take trillion of milleniums. So in order to apply this system to real- 
world application, it is preempted that the input range should be increased and the 
assignment of a number to character be done separately so as to maximize security. 

To clarify Figures 4.1 and 4.2, in order to encrypt, a 16-bit number representing 


a character is partitioned into 4 segments so as to provide the 4 4-bit inputs to the 
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Example of Encrypt/Decrypt of Character 2 
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Note: Message M can only be within a certain range 
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network. Hence the range of M must be sent separately 
via a seperate P.K.S. (RSA). 
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Figure 4.1: Neural Network As A Cryptosystem Block Diagram 
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Figure 4.2: Back-Propagation Network For Encryption and Decryption 


encryption network, the output of which is a single 16-bit number different than that 
of the original input. These 4 4-bit inputs along with their corresponding 16-bit 
output are first fed to the network to train the weights. Once trained, the weights of 
the encryption unit would have converged to values such that when these converged 
weights are set as constants, the same 4 4-bit inputs used for training will provide 
an actual output that can be rounded to the desired output used in training. For 
example, if the desired output is 1256 then the actual output must be between 1255.5 
and 1256.5 so that rounding to the nearest integer would yield 1256. 

Naturally, for a system encrypting up to 45 separate characters, the aorrespond: 
ing training sets will be 45 input/ouput pairs. Basically, this is how the network is 
trained and utilized for encryption. It should be noted that milether the input/output 
pairs are linearly related or not, the weights should converge and accommodate the 
required mapping function. _ 

For decryption, the same type of network, training and mapping scheme will 


be used, only this time the recovery of the original data is essential. Intuitively, the 
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input of the decryption unit is the 16-bit output of the encryption network. To keep 
the structures of the encryption and decryption networks identical, the encryption 
output must be partitioned into 4 4-bit segments before it becomes inputs to be 
decrypted. The desired output of the decryption network must then be the original 
16 bit input of the encryption network. To clarify the process, the followin example 


is offered. 


Example A: 
Given a single processing element with 4 inputs and one output. 

The element’s function is f(S->) = Y; 

The four input x’s= [1 2 A 6]ig ; output=12599 = 313716 

The four converged encryption weights are found to be [77 1056 501 900] such 
that 


1(77) + 2(1056) + 10(501) + 6(900) = 12599. 


The encryption weights are thus : [77 1056 501 900}. 
Since the encrypted output is 313716, the decryption input is (31 3 Tlie 


The four converged decryption weights are found to be [290 66 997 121] such 
that 


3(290) + 1(66) + 3(997) + 7(121) = 4774 = 12Abig. 


The decryption weights are thus : 290 66 997 121. O 


Based on the example, a training set of several encryption and corresponding 
decryption numbers can be randomly picked to represent any character. A typical 


training set for 28 characters, the upper case alphabet with comma and space, is 


shown in Table 4.1. 
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TABLE 4.1: EXAMPLE TRAINING SET 
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Notably, the assignment scheme of Table 4.1 is monoalphabetic. This is chosen 
strictly for simplicity, not security. The focus of of the neural network is to map a 
number to another then recover it. How the number might represent a character is 
entirely another subject in cryptography. In light of this, using training sets similar 
to Table 4.1, experiments were next conducted to support the proposed theory of 


using neural networks for a cryptosystem. 
B. EXPERIMENTAL RESULTS AND OBSERVATIONS 


In order to accommodate the mapping scheme for the proposed cryptosystem, 
a series of experiments designed to gauge the performance of the back—propagation 
network were carried out. The primary goal of the experiments is the development of 
an optimal network based on several parameters. Information such as training time, 
error tolerance, range of input numbers, network sizes and their interdependence 
are of primary interest in building a working example network far the cryptosystem. 
In accomplishing the desired goal, the chosen back—propagation network consists of 
4 inputs, 1 output and 3 hidden layers of various sizes. The network is built and 
simulated using the Neuralware software package [Ref 26] implemented in an IBM 
486, 5SOMHz, 16 Mbytes. 

Table 4.2 provides the first set of results which are intended to show the re- 
lationship between convergence error and training time. For the experiment, a set 
of 45 training input/output pairs (45 characters of NTP) along with 4 bit per in- 
put (16 bit overall since there are 4 inputs) were used. Error is measured in root 
mean squared values (RMS), a common statistical method of error estimation which 
is employed by Neuralware. Training time is compared by number of iterations, a 


method of measurement used in Neuralware. It should be noted that time of iter- 


ations varies for different networks. The larger the network, the time per iteration 
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Number of Elements | Iterations — | RMS Error |} Iterations — | RMS Error 
per Hidden Layer 

































9) 2500 0.6 250000 0.5 

10 73000 0.0025 300000 0.002 
15 70000 0.002 350000 0.00006 
20 124500 0.0005 270500 0.0001 


29 115570 0.000085 340000 0.000017 


TABLE 4.2: TRAINING TIME VS ERROR RELATIONSHIP 


increases proportionally. 

Conclusions drawn from Table 4.2 concern primarily training time and error. 
Comparing the error with iterations to the error, one noted that up to the first set 
of iterations, the errors decreased significantly for all networks. After this, the error 
goes down significantly less even for a greater increase in iterations. This shows 
that after a certain barrier, training of all networks follows the law of diminishing 
return wherein the error decreases minimally despite greater increase in training time. 
Eventually, when the error has reached its minimum, no amount of training time will 
help. This behavior is typical of all neural networks [Ref 24, 25]. After this first 
observation, another set of experiments were run and their results are summarized 
in Table 4.3. For this experiment, the iterations to convergence were set to 3.5 x 10° 
iterations where it was determined that the error was at its minimum for all tested 
networks (weights have converged to optimal values). The inputs again are 4 bit each 
and 45 input/output pairs were used as anne sets. 

Clearly from Table 4.3, given the same set of input/ouput, the larger network 
results in the least error at final convergence. This is due to the larger amount of 
processing elements and weights (memory) available to accommodate the necessary 
mapping patterns. 


The final experiment intends to formulate the interdependence between network 
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Elements/hidden layer 



















5 0.2109 

10 7.835 x 1074 
ea) 3.0836 x 107° 
20 2.492 x 107° 
Za 1.684 x 107° 


TABLE 4.3: RELATIONSHIP BETWEEN NETWORK SIZE AND ERROR 


size, iterations to convergence, and input size. The results are depicted in Figure 4.3. 


The conclusions which can be drawn from Figure 4.3 are: 


e In regards to the range of inputs, as the number of bits per input increases, 
the training time increases. Theoretically, this trend can be attributed to the 
weights having to accommodate mappings of larger number to smaller ones as 
well as the reverse. Namely, as a set of small and large inputs maps to larger 
and smaller outputs respectively, the weights have to be small as well as large 
if there are not enough weights. This may lead to non—convergence as they can 
not be both. This is seen in the extremely high increase in training time with 
the smaller size networks. As the network grows, there are more weights to 


map thus there is less strain on the systern causing training time to decrease. 


e In regards to the number of input/output pairs to be mapped, as the training 
pairs increased to 45 (number of characters in NTP set), the iterations to con- 
vergence also increased. This is easily explained by an analogy to the human 
brain which is the structure emulated by neural networks. When there is more 
information to learn, the brain labors to maximum capacity until its cells are de- 
pleted. In the case of neural networks, as the size of the network is exceeded by 
the information memory’ demands, the iterations increase with approximately 


no learning. A barrier is reached until more neurons are available. 
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Figure 4.3: Relationship between Network Size, Iterations to Convergence and Input 
Size 
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e In regards to the size of the network, the relationship to input/output as well 
as range of inputs are already described in observations of Table 4.2 and 4.3. 
One more observation is added here in that as network size is enlarged for more 
training input or input size, the training time increased. Mathematically this 
makes sense since there are more weights and neurons (memory) to update. 


Each iteration now takes longer to complete. 


After thorough exploration of empirical data, the final conclusion is that there 
exists a network for the proposed cryptosystem. And it works. After several trials, 
the optimal network for this paper’s system is found to consist of a 4 bit per input, 
4 inputs, 1 output, 3 hidden layers, 25 elements per hidden layer, with 45 sets of 
input/output traing pairs. This specific network is used in a conclusive example in 


the next section. 
C. AN IN-DEPTH EXAMPLE 


This example is based on Table 4.1 which in turn is based on the Naval Tacti- 
cal Publication coding scheme wherein a character is mapped unto another: AR, 
BaN... This scheme is chosen for clarity in that an encrypted text will also be a 
string of characters. In reality, however, since the characters are coded by a num- 
ber, the encrypted text need not be a number representing another character. For 
instance, character ‘A’ encrypts to 5BC Fig where 5BC Fj, in this case does not 
represent a character in Table 4.1. 

This example employs a monoalphabetic substitution scheme to assign a number 
to a character. In this respect, this system is vulnerable to single-letter frequency 
analysis and is therefore easy to break [Ref 27]. However, if each character is coded 
by multiple numbers utilizing schemes such as homophonic or polyalphabetic sub- 


stitution (Beale or Vignere and Beaufort cipher), the safety margin would greatly 
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increase [Ref 27]. Additionally, for real-world application, the input range must be 
raised from 16 bit to greater than 256 bit. 

As stated in the previous section, this system, based on RSA, is concerned only 
with two-way mapping a number to another. Bearing this in mind, this section is 
intended only as a pedagogical example of how such a scheme could be implemented 
so as to be able to actually encrypt and decrypt a plaintext message. In reality, 
for complete security, a separate scheme of assigning numbers to characters must 
be: chosen to defeat the frequency of letters in plaintext. If interested, the reader is 
referred to reference 27 for the assignment of numbers to characters. Moreover, the 
range of the network’s input must be greater than 256 bit. Having established the 
objective of this example, illustrations of the system is hereby offered. The following 
plaintext message is encrypted and decrypted using the system of Figure 4.1. 

Plaintext: FIND ME COMPLETE CHAOS AND I WILL SHOW YOU SCI- 
ENCE : 


Decimal coded text and encrypted text: 


F Je . | D M E C 

| | | | | | | 
Plaintext: 21386 | 29325 |38939| 09880103018] 371751 13751|03018| 08523 | 
Encrypted text: 41305 137175121386] 16986 | 54097 | 08523 | 25907 | 54097 | 41525 | 

| | | | i oat | | | 

G M F Ss. xX ¢ Q xX P 


0 M P °L E T E C H A 0 Ss. 


413051 37175/41525| 34609] 13751] 45926 | 13751103018] 08523 26139] 04780/ 41305 | 16986 
04780 | 08523 | 26946 | 30022| 25907 | 62803 | 25907 | 54097 | 41525 |09880| 12828 /04780| 13751 
| —= | | | | | | | | | | 
A C G J Q Z Q x P D R A E 


03018]|04780| 38939 | 09880 | 03018] 29325 | 03818] 54086 | 29325 | 34609| 34609] 03818 | 
64097 | 12828 | 21386 | 16986 | 54097| 37175 | 54097 | 45363 | 37175] 30022130022] 54097 | 
| | | | | | | | | | | | 
K R F S x M x U M J J x 


o1 


16986 | 261391 41305 | 54086 | 03818159988 / 41305 | 4535303818 | 
13751|09880/ 04780 | 48353 | 54097 | 50636 | 04780| 59988 | 54097 | 


16986 | 08623 | 29325 | 13751 /38939|08523/13751 
13751/41525137175 | 25907 | 21386 | 41525 | 25907 
| | | | 
E P M Q 12 P Q 


Resulting encrypted text: 
OMFSXCQXPACG JQZQXPDRAEXRFSXMXUMJ JXEDAUXVAYXEPMQFPQ 


Additionally, given the monoalphabetic scheme chosen here, in order to guard 
against the problem of frequent repetition in the english vocabulary such as the word 
the, double patterns /l, nn, tt which can simplify cryptoanalysis, random or strate- 
gically placed nore can be added to the encryption via some algorithm. Remember 
that since one is using only 28 numbers out of 2)° here, there are multitudes of num- 
bers left to insert into the above patterns as noise bytes. In this specific example, 
the noise is inserted by human intuition and is shown as asterisk (*) signifying any 


number not used in coding the characters. 


An example of encrypted text with noise inserted: 


OMFS *XCQX*PACG* JQZ*QXQDR*AEX*RFSXMXU*MJ** JXE*DAUXV*AYXEPM*QFPQ 


With the noise option, one must have a scheme to filter the noise out prior 
to entering the decryption network. The decryption network simply recover the 
plaintext from the encrypted text as previously discussed. Both the encryption and 


decryption networks is subjected to the following parameters: 
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e Momentum coefficient = 0.300. 
e Learning coefficient = 0.500. 
e Function = Tanh. 


Learning rule = Delta-rule. 


e Size = 4 inputs, 1 output, 3 hidden layers, 25 elements/layer. 
e The time to minimum acceptable error was approximately 8 hours. 


The two networks’ (encryption and decryption) data employed for this example 
are included in Appendix C. 

Clearly, the basis of how to encrypt and decrypt via a neural network is es- 
tablished. Based on knowledge of cryptography, the concept of a key must now be 


incorporated. 
D. KEY MANAGEMENT 


Up until present, the method of mapping has been discussed without any men- 
tioning of a key. In reality, the key evolves from the actual training process. Namely, 
once the training is done, both for encryption and decryption, the converged weights 
are the keys. Since different training sets are used eiverserets): a key for encryption 
and another for decryption are required. The keys will change when the network 
switch mapping function via new training sets. 

For our example of only one training input/ouput pair and one processing el- 
ement in Section A (Example A), the keys are [77 1056 501 900] for encryption and 
(290 66 997 121] for decryption. The fact that two keys must exist is perhaps eer 
now with the example; however, the fact that this is a one-way scheme only remains 


murky. Let’s clarify this further. For a specific set of encryption/decryption key that 


53 


party A obtains from training, party B given the encryption key, can encrypt while 
A can decrypt using decryption key. Unless B somehow also obtain the decryption 
key (the only safe way to do this is through a secured channel) there is no way for A 
to encrypt to B unless B had come up with separate encrypt/decrypt keys of his own 
and sent A the encryption key. There is no restriction against both parties using the 
same encryption/decryption keys that only one has derived, provided the system is 
a secret—key type where the keys can be distributed through safe channels. In this 
respect, there is little to gain from a neural network as it is nothing more than an- 
other mapping method. But there is much more to the versatility of neural network 
which should be exploited. 

In the key management scheme thus far mentioned, only one party needs to 
train the network and then passes the weights as keys for encrypt and decrypt to his 
or her counterpart. However, if both parties were to obtain separate training sets 
and thus keys, only the encryption keys need to be exchanged. In this respect, there 
exists a “pseudo” public-key scheme which can be exploited since the decryption key 


requires no exchange. This possibility is hereby explored. 


1. A Proposed Pseudo Public-Key Cryptosystem Using 
A Neural Network 


Irrefutably in cryptography, the possibility of a péeide-pablieee imple- 
mentation of a neural network merits this paper further examination. Currently, 
the designed networks mentioned that the keys, the Sein tony) decryption weights, 
can be passed through a secured channel. If a cryptoanalyst has the keys and the 
same network, he has broken all codes. Now the assumption is lifted. This research 
postulates that if both parties develop their own set of keys, the encryption keys can 


be exchanged through any public channel(Figure 4.1). A cryptoanalyst having pos- 


o4 


session of the encryption key, a network, and encrypted data will face an enormous 
obstacle in breaking the code: time (in terms of centuries.) 

From the forementioned implementation, one recalls that only the encryp- 
tion key needs to be exchanged if both parties train on separate data and each obtains 
his or her own keys. The decryption key is never divulged. Given the encryption 
key Eencr and the encrypted message Y a cryptoanalyst must solve an excessively 


difficult equation to recover the original input X. 


Example D: 
Using data from our simple one element one input/output training Example A. 
Known to the attacker: Encrypt key (Eencr) and encrypted code. 


7 
1056 
501 
900 


ence = 


encrypted data=313716 


To solve for the original data, he must solve 


172, + 105622 + 501z3 + 90024 = 313716 


with z; being 4 bit, 


which is one equation and four unknown. O 


The above example is done on a simple single processing element model with 
a simple linear function. Given a multilayer network such as the back—propagation 
type with non-linear processing elements, even if the attacker knows the network, the 
problem mathematically increases in difficulty since the number of elements grows 
and thus the amount of required factorizations grows. 

Even with a simple one cell example, for a crude cryptoanalysis method, one 


must solve the equation by trying 2'® combination of inputs to break one character. 


Oo 


Using a crude equation for Table 4.4: 
Time in seconds = 2N4™er of &tt8]99n5(10-° sec computer /loop) 1000computers 


Number of input bits per 2; 


4 (this report’s element) 0.07 ns 
8 4.3 ms 
16 213 days 

32 1.08 x 10!” centuries 
64 3.67 x 10°° centuries 


TABLE 4.4: EXHAUSTIVE SEARCH CRYPTOANALYSIS TIME FOR A SINGLE 
CELL 























On the average it will take less then all combinations as it is probable that the 
solution can come anywhere in the search. An exhaustive search of 2!© loops for 2!° 
combinations poses little arabien, with the power of the computer but let’s say one 
increases the same simple single layer input and output to a 32 -bit, 64-bit , 128-bit, 
or 256-bit input. Herein lies the basis behind the security of this system: a large 
range for the input of the network. Whereas up until now, only 16—bit inputs were 
used in a simple example, when this range is increased to 256 bit, the difficulty of 
working with such a large number renders any cryptoanalysis infeasible. Using an 
exhaustive search, Table 4.4 shows the amount of total possible time it would take 
to break one character given 1000 computers operating at 1 ns per loop operation (a 
very generous, fast time). 

As with all cryptosystems, the time above can be minimized further if the 
system is susceptible to the problem of predictable frequency in the vocabulary. 
Namely, when the number representing trends such as ‘the’, ‘a’, space, double letters 
‘ll, ‘nn‘ exists, estimation of those characters are made easier. With this system, 
there exists a countermeasure in that one could use numbers not mapped to inject 
noise into the transmission thus breaking up any patterns. Here, since only 45 num- 


bers are needed to represent 45 characters, there are 2!© — 45 random numbers left 
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to be used by some algorithm which would insert them into common words such as 
those mentioned above. This possibility was shown earlier in the in-depth example 
of Section C. 

With the multi-element structure of the back-propagation network, the 
cryptoanalysis problem is exponentially greater with increase in number of network 
elements. Undoubtedly, the insurmountable time can be decreased given the luck 
factor in the probabilities and in due time further development in mathematics can 
solve in feasible time the NP complete problem. Nevertheless, at this date, the 


postulate is made that this is a very safe public-key cryptosystem. 
2. Justification of the “Pseudo” Prefix 


Ironically, the restrictions which necessitate the prefix “pseudo” for the 
system arise from the same attributes that make the system safe. Given a range of 
bits of input z, one cannot use all the possible combinations to train the network. 
For example, if each z was 64 bits long, one faces 24%*4 = 27° possible combinations. 
In order to encrypt anything between 0 and 2256 all 27°© numbers must be matched 
to a unique y and trained to the network. This is comparable to the problem of the 
cryptoanalyst; it would take trillions of milleniums — not feasible. 

The solution to this problem is avoidance. One needs only to train a certain 
range of number corresponding to the number of characters needed to be encrypted. 
For the NTP character set in this proposed system, one needs only a range of 45 
out of numbers 2° possible. However, both the encrypter and decrypter must know 
this range. How is this range to be kept a secret and still be passed to both parties? 
In order to make this neural network completely public-key, another PKS system is 
required to pass this range. It is suggested that the already popular Rivest Shamir 
' Adleman (RSA) PKS system mentioned in Chapter II and III be used to pass this 


of 


range. 

In summary, key management involves the direct public disclosure of the 
encryption weights and the indirect public disclosure of the range of inputs via the 
RSA system. This leads to the question of why not use RSA completely and not be 
bothered with the neural network. The answer is that RSA is traditionally slower 
compared to neural networks (after training) and since the range of numbers used in 
encryption/decryption needs to be exchange only once prior to utilizing the system, 
one can afford to use RSA whereas for text encryption, a drawn-out repetitive real- 


time process, a neural network is much more efficient [Ref 12, 24]. 


E. PROBLEMS OF A NEURAL NETWORK AS A CRYP- 
TOSYSTEM AND PROPOSED SOLUTIONS 


The two potentially detrimental problems with the neural network scheme are . 
that of the network weights not converging to an acceptable error for some non- 
linear training sets (non-convergence) and the mapping not guaranteed to be one to 
one (aliasing). Fortunately, the intrinsic versatility of neural networks is such that 
solutions to these problems exist. - 

The more serious of the two problems, non-convergence, can be easily illustrated 
by referring back to the one processing cell, one input/output training set example. 
With simply one cell, an addition of a second input/ouput pair — if not linearly 
related to the first pair —- can cause the cell weights not to converge to acceptable 
errors; namely, there are no possible set ‘of weights which will accommodate the 
correct outputs for both inputs. For example, the input/ouput pair (21 B 6]:¢ and 
[0 E F 3]ig is added to example 4.A. Using the old convergence weight for the original 


input/output, the actual output of the second pair is: 
2(77) + 1(1056) + 11(501) + 6(900) = 12,121 = 2F/5946. 
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Clearly this is not the desired output for the second input. Hence, if one was to 
use the two data set above to train the single cell, the weights would not converge. 
One is then left with some restriction as to how to choose training set (mapping func- 
tion). This restriction, can be easily exploited by a cryptoanalyst to break the system 
as he or she now knows that only certain mapping function is possible given knowl- 
edge of the system. Luckily, this restriction can be lifted with the back—-propagation 
network used in this research. 

As previously mentioned in Section A, a back-propagation network is an excel- 
lent mapping method of non-linear functions. Relying on this property, the training 
sets for encryption and decryption do not need to be linearly related. The more cells 
one adds to the network, the more non-linear functions can be mapped. Theoreti- 
cally, with enough cells per layers, the weights will converge to acceptable errors given 
just any training data (Ref 24]. For the non-convergence example above, indeed the 
back-propagation network did prove to be the solution. 

Additionally for public-key cryptography, one must bear in mind that the train- 
ing data for encryption and decryption are related. For it to work, the weights of both 
encryption and decryption networks must converge. A training set that converges for 
encryption but its inverse training set does not yield converged weights for the decryp- 
tion network is otherwise of no use in cryptography. From experimental data of the 
proposed 45 character encryption/decryption scheme, using the back-propagation 
system, problems of convergence were sometimes encountered. The reader is referred 
back to the experimental Section B where it was shown that when non-convergence 
does surface, the solution is to add more cells. 

Apart from non-convergence, the second problem, aliasing, proved less serious 
but still needed to be dealt with. Aliasing occurs when, given a converged weights, 


two or more sets of inputs map to the same output. This nuisance can be attributed 
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to the same problem which necessitated the “pseudo” prefix. Since one trains only 
a range of inputs within the vast possibility (> 27°°), the unused inputs could by 


chance map to one of the same chosen outputs. 


Example E: 
Again reverting back to the one cell, one input/output training set of Example A in 
Section A, an input of [1 2 A 6];¢ along with encryption weights of [77 1056 501 900] 
yielded an encrypted code of 12599 = 313746. 

Let’s use an input of [714 A]j¢ and the same converged weights. The encrypted 


code for this input will be 
7(77) + 1(1056) + 4(501) + 10(900) = 12599 = 3137.6, 


which is the same output with the original input; hence aliasing has occured. O 


Clearly aliasing is a theoretical possibility and thus a problem; however, in real- 
ity it can be easily be avoided by making sure one uses only the trained input /output 
pairs for encryption and decryption. This way, one knows exactly that a given en- 
cryption output should map back to the desired encryption input during decryption 
and not the aliased value. In fact, the alias problem can be exploited to the system’s 
advantage. If certain aliasing problems are adapted intentionally, cryptoanalysis be- 
comes more difficult. As previously explained in the “pseudo” justification section, 
only the desired parties knows the range of inputs to use whereas others do not. It 
is essential only to choose exact one-to-one mapping pairs in this range to avoid 
aliasing. Outside this range, any other inputs can have the aliasing effect, an actual 


benefit in extra safety. 
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F. DEVELOPMENT OF A COMPLETE BLOCK-DIAGRAM- 
LEVEL HARDWARE SCHEME USING A NEURAL NET- 
WORK IN PKS 


Up until now, most of the basic building blocks of a PKS using neural network 
have been discussed. Gathering all the essential blocks together, a possible block 
diagram proposal for an entire cryptosystem is shown in Figure 4.4. 


Block by block description of Figure 4.4. 


e The only component not yet delved into is the automatic generator of training 
input/ouput sets. This function can be fulfilled by a linear feedback shift 
register (LFSR). Given an input polynomial, it is a simple circuit capable of 
generating a random set of different numbers given. For this study, an LFSR of 

' order 16 ee to generate 2!°—1 random numbers for both input/output 
pairs of encryption. For further insights on LFSR’s, consult reference 28. After 
the input/ouput training sets of encryption is established by the LFSR, the 
decryption input/ouput training sets must be the inverse; namely ouput and 


input of encryption become input and input of decryption, respectively. 


e Decrypt/encrypt neural net- Both networks are of the back-propagation type 


composed of 4 inputs , 1 ouput, 3 hidden layers with 25 elements per layer. 


e Input Range Exchange- As discussed in Section D.2, the RSA hardware of 
Chapter III can be used to send the range thus making this a “pseudo” PKS. 


e Network Weights- The weights of the neural networks Rest be able to undergo 
changes during training and then be set to constants once the the converged 
weights are obtained via training or received from opposite parties. Simple 

- latches and switches seem adequate for the task although no detail studies are 


made. 


61 


Linear Feedback Register 
Encryption (Generate Training Sets) _ 





Xx (training) Yk 
Receive Ranga_” 
Rang RSA 
ge 
Ik PKS Machine Receive |Encrypt Here 


x 


Wea—"Public Key 
Send Encrypted 
Message Here 


7 | 
a= ? — 
Encrypt | Y — 
Training 


Encrypted 
Message 


Encrypt 


Truncate 
To X:...Xa 







Decrypt 
Network 


Yde_k 
Message Here 





Private Key ~*—— Waec 


Figure 4.4: Neural Network in PKS 


62 


Receive Encrypte 


ee EEE ——— ee 





A working model of a public-key cryptosystem based on neural networks has 
been designed. It is merely a sample model which can be applied in limited usage; 


however, the idea behind the system deserves recognition as a worthwhile alternative 


to PKS. 
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V. CONCLUSION 


This thesis has presented some novel approaches to public-key cryptosystems. 
The focus was centered on a specific hardware implementation and a completely 
new angle to PKS using neural networks. In both issues, research produced working 
models when simulated by computers. 

The hardware implementation for a modulo reduction unit in a fast exponentia- 
tor — an essential device in the most popular PKS, RSA cryptosystem - was developed 
based on the sum-of-residues method (SOR). The design is based on the concept 
of modularity. The modular unit can be conveniently connected to form a fast ex- 
ponentiator for numbers of any length. The result is a working VLSI layout when 
simulated by RNL ( Rppendr C). The efficiency in speed and size, though offered in 
the study, remains issues to be considered when the unit is to be used in real-world 
applications. If the speed and size given hereby are acceptable to a certain applica- 
tion then this unit is perhaps a viable alternative to existing technology due to its 
advantage in modularity. | 

The second part of this thesis involves the use of neural networks in PKS. To 
the author’s knowledge, the attempt to integrate neural networks into cryptography 
is a novel idea. Whether it is either original or even revolutionary remains to be seen. 
That the goal is at all plausible is an unanticipated surprise when the experimental 
results confirmed it so. This is not to say that plausibility means practicality. So far, 
all that is proven is that the concept works. Whether the scheme is feasible needs 


further research. 
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From data gathered in Tables 2.4 and 4.4, one can conclude that at 256 bit 
in length for the key in RSA and input in the neural network-based cryptosystem, 
exhaustive cryptoanalysis faces infeasible time limit. For all practical purpose, re- 
quiring trillion of milleniums to break, the system of this thesis is as safe as any 
current PKS (Table 4.4). Additionally, the most significant advantage in using neu- 
ral networks in PKS is that there is no need for fast exponentiation which has proven 
to be slow for large exponents and modulus [Ref 2]. The only necessary operations 
in a back-propagation network are multiplication, addition and hyperbolic tangent 
(or other non-linear functions.) The computational feasibility of the neural network 
scheme, however, is not explored here and is left to follow-on research. 

At present, the example system only applies for input ranging 16 bit in length. 
For the system to be secured, it is suggested that the range be extended to 256 bit. 
Intuitively, if one single network is to be used to map numbers with 256 bit range, 
it will have to be large and thus will slow down the system. However, if parallel 
processing is available and one can afford to design a 256 bit cryptosystem based on 
16 16-bit neural networks, the results of this paper will be of value. Furthermore, 
only the back-propagation network was used in this research. Given the multitudes 
of network types in various applications, there may exist other schemes capable of 
using other networks. 

This paper is intended to pioneer the idea of neural network in cryptosystem. 
As such it claims only the initiative in a novel avenue to cryptography. The proposed 
theory of employing neural networks in Eton ache now ends with a call for further 
research into the efficiency, speed and possibilities of more capable networks. The 
key to the knowledge gathered so far is that a new method is postulated and there 
seems to be some merit in that it works with some restrictions. These restrictions 


may be lifted by further investigation or perhaps there shall come a disproval which 
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may destroy the entire scheme altogether. Be that as it may, time constraint dictates 
that this introductory study terminates with many aspirations of fueling follow-on 


research in this subject. 
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APPENDIX A 
SUPPLEMENTARY PROGRAMS 


The following programs are provided to supplement background knowledge in 
public-key cryptography. In order, they are: fast exponentiation, greatest common 
divisor, inverse, and factorization. The first three programs are written in C [Ref 
2] and run on Unix while factorization is in Matlab code and ran on an IBM ’486, 


50MHz, 16MB. 


He Ae He He he Ae AC iC 2K 2h ae he 2c IC ake 2c Ie 2c aie he aie he aie he he aie he he A he aie A Ae aie Ae aie he he hehe aie hehe Ae fe 2A Ache 2A he 2 ie oie fe ake oie aie fe Ae afc Ie af oie ofc 2c ok ok 


/* 
This program uses the fast exponential algorithm to compute the operation: 
a~Z mod n. It 1s intended as an example of software implementation of the 


RSA public key cryptosystem. */ 
#include <stdio.h> 


/* The algorithm is contained in the following function to be called when 


necessary. */ 


int fastexp(a, z, n) 
ima, Z,n; 
{ 

mitex = 1; 


while (z) 
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while ('(z % 2)) 


{ 

z /= 2; 

a = ((adn)*(a % n)) An; 
Za; 


x= ((x 4 n)*(a %4n)) 40; 


} 
return (x); 

i; 

main () 

al 
INntcawazeen ee: | 
printf("a°z(mcd n). Enter a) zane); 
scanf("%d %d %d0" ,&a,&z,&n) ; 
t= fastexp( a, z, n); 
printf("Result = %d\n", t ); 

I; 


EE He ee HE HE A He Ae A He he A I i 2h 2h 2 2 he fe 2 RK 2 2 2h 2 ic 2c 2 2c 2c AE ic 2c IE 2c a 2 a CC I A eC i i 2K 2c ok 2 


/* 
This program uses Euclid’s algorithm to solve for the greatest common 
denominator (gcd) of two number. Given two input integers, a and n, this 


program provides their mutual ged. This is intended to be an example for 
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generating keys in the RSA public key system */ 


#include <stdio.h> 
main() 
i 
int g[100]; /* Initialize an array for gcd */ 
ant 1=1; 
printf ("gcd of a,n. Enter a,n separated by space:") ; 
scanf ("Y%d %d0", &g[0], &gli]); 
while (g[i]) 
{ 
gCi+i] = gfli-1] % glil; 
1++; | 
} 
preantt ("gcd of %d and %d is %d \n",g(0],g¢[1],g[i-1]); 


We He He He Ae Ae He he Ae He Ae he 2H 2 he 2c 2A 2 he 2 2c ae 2h 2 2 2c 2c 2c he he 2c 2 2 ie 2c 2c 2c fe 2c 2c ie ae 2h fe eae 2c 2c ie ee 2c ee oe ee 


/* This program compute the inverse, x, of a and n (0<a<n) such that 


ax (mod n) = 1 */ 


#include <stdio.h> 


main() 
if 


int g[100], uf[100], v{100]; /* Initialize arrays for indexing */ 
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int i=1; /* Beginning index # of loop */ 
Intayn a. /* Defining input and intermediate var. */ 
printf ("inverse of a,n. Enter a,n separated by space: "); 
scanf ("Yd %do", &a, &n); /* Read in a and n */ 
glO]= n; | 
glil= a; 
ufO]= v{i] = 1; 
uli] = v[0] = 0; 
while (gfil) 
{ 
glijJ= uli] * n + v[i] * a; 
y= gli-1i)/glil; 
gCit+i] = gfli-1] - y*g[il; 


ufi+i1] = ufi-1] - y*u[il; 


v(Cit1] = v({i-1] - y*v(il; 


i++; 


} /* Using extension of Euclid’s gcd algo */ 
if (v[i-1] <= 0) 


{ 

printf ("inv of %d and %d is %d \n", a,n,v[i-1i]+n); 
} 
else 
{ 

printf ("inv of 4d and wd is %d \n",a,n,v[i-1]+2*n) ; 
} 
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PSST LESS S SS STS SESE TSE TE LESS TELE ETE SETTLES SE ETE STEELE SE TELE TES ELST SS SS 


74, This is a Matlab program designed to factorize a product of two 

% primes for the cryptoanalysis of the RSA public-key cryptosystem. 
% Intended merely to show the futility of factorizing large numbers, 
% it employs a naive exhaustive search method of dividing and 


7% checking the remainder of the division of the product and every 


> 


possible odd numbers until a factor is found. To use the progran, 


D4 


simply type rsafac(‘product of 2 primes’). 


function [x]=rsafac(z) ; % Enter the product. 
w=round(sqrt(z)); 74 Factor can not be larger than 


% the square root of the product. 


for n=1:2:w 7%, No need to test even numbers, and 
% limit of search is w. 
v=z/n; 4 Testing by dividing products by 
% odd numbers. 


if (rem(v,1)==0) 4 If v is integer then 


x=(n,v]; %n and v are factors. 
n=W; % Exit loop once factors are found 
end 
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APPENDIX B 


RNL SIMULATION OF MODULO 
REDUCTION UNIT 


The following examples are indicative of the successful RNL simulation [Ref 3] 
of the final modulo reduction unit. The unit simulated here is limited to modulo 
numbes of 4—bit length. The RNL control file, stimulation file for one example are 


included along with simulation results of 5 modulo operations. 


Sample control file for RNL simulation of 5 mod 7 using modulo reduction 
layout of Figure 3.11. 

; The name of this control file for rnl is: modi.l 

; Simulation for modulo reduction unit of Chapter 3. 
; LOAD STANDARD LIBRARY ROUTINES 

(load "uwstd.1") 

(load "uwsim.1") 

; FILE WHICH WILL LOG THE RESULTS 

(log-file "mod1.rlog") 

; READ IN THE BINARY NETWORK FILE 

(read-network "modi") 

; DEFINE THE TIME SCALE FOR SIMULATION 

(setq incr 90) 

; DEFINE INPUT VECTOR IF ANY, standard STYLE 


(defvec ’(bit state s3 s2 31 s0 )) 


(2 


- DEFINE INPUT VECTOR IF ANY, SINGLE INDEX STYLE 

- DEFINE INPUT VECTOR IF ANY, double index STYLE 

- STANDARD REPORT FORMAT DEFINITION. 

(def-report ’("response= " cli cl2 in i3 i2 i1 (vec state))) 
PEEL OUSILE SPECIFIED 

openplot "modi.beh" 

; LOGIC ANALYZER STYLE OUTPUT FORMAT SELECTION. 

(setq lanalyze t) 

(wr-format) 

meeGuilcH DETECTOR SELECTION. 

(setq glitch-detect t) 

; NODE TRANSIENTS REPORT DEFINITION. 

(chflag ’( s3 s2 s1 s0)) 

; TRIGGER CONDITION SET-UP 

; ADDITIONAL SIMULATION SET-UP COMMAND LINES. 

(printf "Commence simulation...\n") 

; SPECIFICATION OF A TIME/BASENAME FILE FOR INCLUSION. 
(load "mod1.time") 

; ADDITIONAL WRAP-UP COMMAND LINES. 

(printf "...completed simulation! \n") 

exit 

; GEN-CONTROL COMPLETED. 

We Ae Ae He HE ie 2c he he ie Ie A i ie he ie ac i i eR 2c he ie fe ic hc ic 2 ie fee fe ee 2c he fe AK ee fee AC A he he ht ae fe i 2 i 
;The following is the stimulation file for the input to the rnl simulation 
;sabove for 5 mod 7. 


Sample < >.stim file for 5 mod 7: 
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time_range 0 10 
inQhoizaé 
immnO1lOA2ZI1 4 


eli 20) ent 
élin 2h O lei 


ClZa2  Uaier 


ClZn 2°10 Bo? 


opt OhO x il 


optn 0 10x 1 


m0 
mi 
m2 


m3 


33 
32 
si 


sQ 


13 
12 


Abt 


oOo oOo 


© 
yr f- eK Pp 


MPP FP 


- Note 101 is entered for 5 


; Simply inverse of in 


; 2-phase clocks 


; Initializing MUX select 


; 2’s complement of 7 is 1001 


; Modulo number inputs 


; Initializing summer 


; Initializing ist residue to 1 


14 


report 1 0 


PEE SSL ESE SESE SE SELES EE SESE SEE TEEPE SES ETE PTET TSE ET ET TES ET ETT TTS ES SS 


;The following is the RNL simulation result of stimulation file above 
> mod / : 
; 118 nodes, transistors: enh=68 intrinsic=0 p-chan=56 dep=0 
; Low~power=20 pullup=0 resistor=0 
; Report format of logic analyzer style output 
fame cli ¢l2 in eoe2, 1 fT state(result) 
* a2 


Commence Simulation... 


2) Diy 1 i7  . 00 1 0000 

18 i 0 1 001 0001 - ist clk pulse 

27 oO 1 0 010 0001 

36 1 0 0 010 0001 - 2nd clk pulse 

45 oe 1 sf 100 0001 

54 i «0 1 100 0101 - 3rd clk pulse *** 
63 oe a i ONC! 0101 


...completed simulation! 
* Input is 101= 5 (Note input taken at each rising clock edge.) 
** Residues are 1,2,4,1,2,4... for mod 7. 


*** 5 mod 7 = 0101= 5. 


ee ee RR RII RO oO I I Ok ik iC ok keke ok ake ake ke ke 2c 
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;The following is a second RNL simulation result (10 mod 6): 


e 
3 


118 nodes, transistors: enh=68 intrinsic=0 p-chan=56 


;dep=0 low-power=0 pullup=0 resistor=0 


; Report format of logic analyzer style output 


time 


Commence simulation... 


9 

18 
Zi 
36 
45 
54 
63 
72 


...completed simulation! 
* Input is 1010= 10. 
** Residues are 1,2,4.. 


***x {0 mod 6 = 0100 


Crile? 


0 


pl 


0 


1 


1 
0 


in 


a 


0 


0 


1 
iL 


i3 i2 i1 state(result) 


28 

00 i 0000 

00 1 0000 - ist clk pulse 
OU 0000 

010 0010 - 2nd clk pulse 
100 0010 

100 0010 - 3rd clk pulse 
010 0010 

010 0100 - 4th clk pulse *** 
, LORemMod ai. 


He He he he he he he he he ah he he 2 2A he ee ie ee fe 2h 2c ae hc ee he ah hc et ae hc i hc ie ie hc hee he he he hc he he he Ae he ec he ee ie ec i hc i ee i 


;Third RNL simulation using 10 mod 7: 


; 118 nodes, transistors: enh=68 intrinsic=0 p-chan=56 


; dep=0 low-power=0 pullup=0 resistor=0 


16 


; Report format of logic analyzer style output 
time cli cl2 in Poe a state(result) 


* OS 


Commence simulation... 


9 So 1 0 O01 0000 

18 1 0 0 001 0000 - ist clk pulse 

27 oF 1 1 010 0000 

36 iO 1 010 0010 ~- 2nd clk pulse 

45 oO 1 0 100 0010 

54 i 0 0 100 0010 - 3rd clk pulse 

63 a 1 1 001 0010 

72 1 0 i 001 0011 - 4th clk pulse *** 


...completed simulation! 
* Input is 1010= 10. 
** Residues for mod 7 is 1,2,4,1,2,4... 


**10 mod 7= 0011 = 3. 


Pee Pe S STE SEES SS EEE TS SE PELE SEPT E PES ESTES EEE ETE LE SE TSE SP EES ETE SSS SS FS 


; Fourth RNL simulation using 11 mod 6. 
; 118 nodes, transistors: enh=68 intrinsic=0 p-chan=56 
; dep=0 low-power=0 pullup=0 resistor=0 
; Report format of logic analyzer style output 
time cli cl12 in peste lytA state(result) 
* ek 


Commence simulation... 


v7 


2 

18 
27 
36 
45 
94 
63 
TZ 


81 


...completed simulation! 


* input is 1011= 11. 


u 


0 


0 


1 


iL 


001 0000 


001 0001 - ist clk pulse 
Ui 0001 

oh 9) 0011 - 2nd clk pulse 
100 0011 

10 0 0011 - 3rd clk pulse 
010 0011 

010 0101 - 4th clk pulse**« 
(8) (0, 0101 


**x Residues of mod 6 are 1,2,4,2,4... 


*x**x 11 mod 6= 0101= 5 


Ae he ae he he he ae he ae he ae fc fe hc ake ie a ke ef ie ae af ke ae akc he ah 2c fe tfc ie ae ie ake af 2 fe kc afc fe etc ke afc akc fe fc he ake ae oe ac hc ke ac oe ae 


; Fifth RNL simulation with 17 mod 5 


; 118 nodes, transistors: enh=68 intrinsic=0 p-chan=56 


; dep=0 low-power=0 pullup=0 resistor=0 


; Report format of logic analyzer style output 


time 


Commence simulation... 


S 


18 


GliieGiZ 


0 


1 


1 


0 


mise 


b 3 


1 


1 


13° 12920 state(result) 


4 
001 0000 
CFC rt 0001 - ist clk pulse 


78 


27 0 1 0 010 0001 


36 1 0 0 Om 0 0001 
45 0 1 0 10 0 0001 
54 1 0 0 10 0 0001 
63 0 1 0 Orme 0001 
16? mm «(OO 0 oi ae 0001 
81 0 bl i OOo 0001 
90 1 0 1 ORO 0010 
go 0 1 1 010 0010 


...completed simulation! 


* Input is 10001= 17. 


** Residues of mod 5 are 1,2,4,3,1,2,4,3... 


*** 17 mod 5=0010 = 2. 
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2nd clk pulse 


3rd clk pulse 


4th clk pulse 


oth clk pulse*** 


APPENDIX C 


SAMPLE NEURAL NETWORK FROM 
NEURALWARE 


The following is data for the encryption and decryption neural network used in 
Chapter IV in-depth example. The network data is formatted from Neuralware [Ref 
26] “annotated” option once convergence is reached. This option piovides all the 
necessary parameters to reconstruct the network trained by data from Table 4.1. Of 
the many parameters, those of interest are learning iterations (375642 for encryption 
and 333877 for decryption), error function ( standard = hyperbolic tangent), learning 
rule (delta—-rule), and the processing elements’ data. Of the element’s data, the error 
for each element’s output was approximately zero once convergence is reached. The 
weight data are not included other than the number of weights going to each element. 
The reason for this omission is that it is not pertinent. With the data offered here 
and Table 4.1, one can reconstruct the encryption and decryption network using 


Neuralware. 


Title: Encryption Network for In-~Depth Example ; 
Display Mode: Network Type: Hetero-Associative 
Control Strategy: backprop L/R Schedule: backprop 


375642 Learn QO Recall QO Layer 
16 Aux 1 O Aux 2 O Aux 3 
L/R Schedule: backprop 
Recall Step ie 0 0 0 0 
Firing Density 100.0000 0.0000 0.0000 0.0000 0.0000 
ae Tisaae 0.0000 0.0000 0.0000 0.0000 
P 0.0000 0.0000 0.0000 0.000 
Learn step 5000 0 0 0 0 7 
Coefficient HM 0.9000 0.0000 0.0000 0.0000 0.0000 
erg ; near 0.0000 0.0000 0.0000 0.0000 
oe clen .0000 0.0000 OF 
PR 0000 0.0000 0.0000 
Learn Data: File Rand. (Encryption file here) Binary 


Recall Data: 


File Seq. (Encryption file here) 
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Result File: 


Desired Output, Output 


UserIO Program: userio 
I/P Ranges: ie OOUOF 1.0000 
O/P Ranges: =O SOOO, 0.8000 
I/P Start Col: 1 MinMax Table: sama 
O/P Start Col: 5 Number of Entries: 5 
MinMax Table <sama>: 
Col: 1 2 z) 4 5 
Min: 0.0000 1.0000 1.0000 120000 2445 .0000 
Max: 15 Lab 12 14 6.28e+004 
Layer: l 
PEs: 1 Wgt Fields: 2 Sum: Sum 
Spacing: 5 F' offset: 0.00 Transfer: Linear 
Shape: Square Output: Direct 
Scale: 1.00 Low Limit: —-9999.00 Error Func: standard 
Offset: 0.00 High Limit: 9999.00 Learn: --None-- 
ie Low: —0.100 gps Iebeejeiy (a). tMare, L/R Schedule: (Network) 
Winner 1: None Winner 2: Ncne 
PE: Bias 
1.000 Err Factor 0.000 Desired 
0.000 Sum 1.000 Transfer 1.000 Output 
QO Weights aoe 20 EEror 0.000 Current Error 
Laver: In 
PEs: 4 Wgt Fields: 1 Sum: Sum 
Spacing: 5 F' offset: 0.00 Transfer: Linear 
Shape: Square Output: Direct 
Scale: 1.00 Low Limit: -9999.00 Error Func: standard 
Offset: 0.00 High Limit: 9999.00 Learn: ~--None-- 
Init Low: -0.100 ade Heche On LOO L/R Schedule: (Network) 
Winner 1: None Winner 2: None 


PE: 2 


1.000 Err Factor -0.867 Desired 
Zao.560/ Sum -0.867 Transfer =O. 867eoutput 
xxx Q Weights C000 Error: 0.000 Current Error 
xxx From here on all error for all PE's are 0's. 
PE: 3 
mOO0 Err Factor -0.800 Desired 
-0.800 Sum -Q.800 Transfer aO.cOQ CUutpUut 
PE: 4 
me000 Err Factor 0.636 Desired 
0.636 Sum 0.636 Transfer 0.636 Output 
PE: 5 
1.000 Err Factor 0.692 Desired 
o2692 Sum 0.692 Transfer O2692 Output 
Layer: Hiddenl 
PES: 25 Wgt Fields: 2 Sum: Sum 
Spacing: 5 F' offset: 0.00 Transfer: TanH 
Shape: Square Output: Direct 
Scale: 1.00 Low Limit: -9999.00 Error Func: standard 
Offset: 0.00 High Limit: 9999.00 Learn: Delta-Rule 
ime Low: —0.100 Init High: 0.100 L/R Schedule: hiddenl 


Winner 1: None 


Winner 2: None 


wR Schedule: hiddenl 


Recall Step a 0 0 0 0 
Firing Density 100.0000 0.0000 0.0000 0.0000 0.0000 
Gain 1.0000 0.0000 0.0000 0.0000 0.0000 
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kkk 
xxx 


Gain 


Learn Step 


Coefficient 1 
Coefficient 2 
Coefficient 3 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


6 


1.000 Err Factor 


0.044 


Weights 


Sum 


1.0000 
10000 

0.3000 
0.3000 
0.1000 


QO. 
Q. 


0.0000 0.0000 0.0000 
30000 70000 150000 
0.1800 0.0648 0.0084 
0.1800 0.0648 0.0084 
0.1000 0.1000 0.1000 
000 Desired 
044 Transfer 0.044 
0.000 


0.000 Error 


From here on all weights 


.000 
OZ 


ZS 


.000 


5 
IL 
¢) 
8 
1.000 
0 
9 
il 
0.500 


Err 
Sum 


BEE 
Sum 


Err 
Sum 


Err 
Sum 


Err 
Sum 


je ag 
Sum 


Ere 
Sum 


Da ah a 
Sum 


jy ed 
Sum 


Err 
Sum 


)a a 
Sum 


Ihe g 
Sum 
Err 
Sum 


jpeg 
Sum 


Da on a 
Sum 


Factor 


Paceon 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


FaGcreor 


Facvor 


Factor 


Paceor 


‘Oh 
QO. 


! | { | { { 
OO OO OO OO OO Oo OO OO OO OO OO COO OO O O 


are 


000 
546 


.000 
223 


.000 
-462 


.000 
BOOZ 


.000 
.069 


. 000 
. 144 


.000 
.008 


.900 
~290 


.000 
.045 


.900 
Hey, 


.000 
.037 


.000 
Seas) 


.9000 
.023 


000 
224 


. 000 
rood 
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5S and errors are QO. 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


0.546 


. tes 


0.462 


. Jann 


.069 


0.144 


.008 


296 


.045 


. 3a 


.037 


ons 


0.023 


0.224 


oor 


0.0000 
310000 
0.0001 
0.0001 
0.1000 


Output 


Current Error 


Output 


Output 


Output 


Output 


Queue 


Output. 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


1.000 Err Factor 0.000 Desired 
1.274 Sum 0.855 Transfer Ohateeie) (Oh eyeyb ts 
PE: 23 
imo0O Err Factor 0.000 Desired 
o.031 Sum 0.031 Transfer 0.031 Output 
PE: 24 
1.000 Err Factor 0.000 Desired 
0.029 Sum 0.029 Transfer OR ZoeOue put 
PE: 25 
meooO Err Factor 0.000 Desired 
0.816 Sum 0.673 Transfer 0.673 Output 
PE: 26 
moo0O Err Factor 0.000 Desired 
-0.286 Sum On 27 Jetranster SOAS: Whee); 2 
PE 27 
meoo0 Err Factor 0.000 Desired 
=On299 Sum -0.290 Transfer -0.290 Output 
PE: 28 
imo0O Err Factor 0.000 Desired 
1.650 Sum 0.929 Transfer 0.929 Output 
PE: 29 
mo00 Err Factor 0.000 Desired 
0.891 Sum 0.712 Transfer 0.712 Output 
PE: 30 
1.000 Err Factor 0.000 Desired 
0.440 Sum 0.414 Transfer 0.414 Output 
Layer: Hidden2 
PES: 25 Wgt Fields: 2 Sum: Sum 
Spacing: 5 Ee Voneser-) 0. 00 Transfer: TanH 
Shape: Square Output: Direct 
Scale: 1.00 Low Limit: -9999.00 Error Func: standard 
Offset: 0.00 High Limit: 9999.00 Learn: Delta-Rule 
inet Low: ~0O.100 opie Jebroey Ley alleye L/R Schedule: hidden2 


Winner 1: None 


Winner 2: None 


L/R Schedule: hidden2 


Recall Step AF 0 0 0 0 
Firing Density 100.0000 0.0000 0.0000 OZOUCo 0.0000 
Gain 1.0000 0.0000 0.0000 0.0000 0.0000 
Gain 1.0000 0.0000 0.0000 0.0000 0.0000 

Learn Step 10000 30000 70000 150000 310000 
Coefficient 1l O25 00 0: 1500 0.0540 0.0070 0.0001 
Coefficient 2 0.3000 0.1800 0.0648 0.0084 0.0001 
Coefficient 3 0.1000 0.1000 0.1000 0.1000 0.1000 

PE: 31 

imoug Err Factor 0.000 Desired 
2221. Sum 0.218 Transfer On 2s sOueput 
*x*26 Weights =0 OOCP Error =O, COOVCUrrent Error 


xxx From here on all PE's have 26 


PE: 32 


muO0O Err Factor 0.000 
-1.459 Sum ORE sil 
PE: 33 
moO Err Factor 0.000 
-2.230 Sum =O o7 7 
PE: 34 
a-O000 Err Factor 0.000 
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weights, approximately 0 error. 


Desired 
Transfer =0).897 Output 
Desired 
Transfer e069)  MOUE DUE 
Desired 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PEt 


PE: 


PE: 


PE: 


Oe on 
35 
sl 16,10,10 

-0.168 
36 
1.000 
Os bs. 
oy 
ESTO) 040 
L252 


1.000 
=O 65 

39 

F000 
-1, 256 


pm olele) 
=O 20 

41 

i000 
ake Ae 


1.000 
Ze50. 


1.000 
0.082 


1.000 
mers eeyone 
45 

1. COC 
4.263 


1.000 
= OL beh 
47 
1.000 
wo. OCs 
48 
1.000 
=O5707 
49 
1.000 
mO 25 2ih 


L000 
gece eS 


1.000 
pe eae, 
52 
1.000 
0.934 
a5 


Sum 


Err 
Sum 


Err 
Sum 


Err 
Sum 


Err 
Sum 


Err 
Sum 


Err 
Sum 


Err 
Sum 


Err 
Sum 


Err 
Sum 


|p ap 
Sum 


Err 
Sum 


Err 
Sum 


| Jag 
Sum 


Ecr 
Sum 


Sieg @ 
Sum 


Err 
Sum 


Err 
Sum 


Err 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor : 


Factor 


Factor 


Factor 


Factor 


Factor 


FacTeor 


Factor 


accor 


Factor ° 


i 
© 


{ | ( J i 


O © OO CO © OO OO O © OO Fo OO OO OO © O OO OO OO OO OO OO 


. 288 


000 
L167 


- 000 
05 


000 
. 818 


000 
- 164 


.000 
.850 


. 000 
~477 


. 900 
noo? 


000 
-205 


- 000 
2Ocr 


000 
Be) 0 


- 000 
.000 


- 000 
= hee 


. 000 
068 


. 000 
lo 9. 


. 000 
.483 


000 
a Wi 


.000 
a 0 


000 
735 
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Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 


Transfer. 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


2G 


167 


305 


yore 


- 164 


.850 


477 


Boy)! 


O78 


088 


B= 2h0, 


.000 


156 


.068 


Bos ORS, 


. 483 


. Foe, 


« Hal 


735 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


QuErPUt 


Output 


Output 


PE: 


PE: 


1.000 
=o. 033 
54 
1.000 
-2.768 
5) 
1.000 
go 017 


Bees 
Sum 


Ber 
Sum 


Err 
Sum 


/Layer: Hidden3 
PES: 25 
Spacing: 5 


Factor 


Factor 


Factor 


.000 
.033 


OCG 
Bess 174 


.000 
Pol 7, 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Wgt Fields: 2 
' offset: 0.00 


2 


Shape: Square 


Scale: 1.00 
Offset: 0.00 
Init Low: 


-0.100 
Winner 1: None 


Low Limit: 


a2999...00 


High Limit: 9999.00 


L/R Schedule: hidden3 


Recall Step 


Firing Density 0 


Temperature 
Gain 
Gain 
Learn Step 
Coefficient 
Coefficient 
Coefficient 
PE: 56 


PE: 
PE: 
PES: 
PE: 
PE: 
ele 
PB: 
PE: 


PE: 


OOO 
0.421 
57 
1.000 
morn 212 
58 
oO 
0.145 
Be, 
1.000 
eo. 139 
60 
1.000 
m@. 209 
61 
1.000 
0.137 
62 
1.000 


) hep 


63 
i000 
mou. 306 
64 
1.000 
0.669 
65 
1.000 
S(O ps 


Err 
Sum 


Meg 
Sum 


Err 
Sum 


EEG 
Sum 


Beis 
Sum 


per 
Sum 


prr 
Sum 


Er 
Sum 


Err 
Sum 


lSieig 
Sum 


x 


2 


3 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


J 
0.0000 
0.0000 
1.0000 
1.0000 

10000 
Q.2000 
O25000 
0.1000 


i 
OO OO Oo OO OO OO OO OO OO OO 


Init High: 0.100 


0 
0.0000 
0.0000 
0.0000 
0.0000 

30000 
aE zZo0 
0.1800 
0.1000 


Error Func: 


20700355 


BO 92 


Oroly 


Sum: 


Transfer: 


OUEDUC: 


Output 


Output 


Output 


Sum 
TanH 
Direct 


standard 
Learn: Delta-Rule 


L/R Schedule: hidden3 


0 
0.0000 
0.0000 
0.0000 
0.0000 

70000 
0.0432 
0.0648 
0.1000 


.000 
x98 


.000 
GAG, 


000 
.144 


.000 
esS 


.000 
. 206 


.000 
7136 


.000 
sale O 


.000 
; oon 


.900 
. 984 


.000 
wus 
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Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 


Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


0 
0.0000 
0.0000 
0.0000 
0.0000 
150000 
GZ0C56 
0.0084 
0.1000 


Ore Sieh 


= One 


0.144 


sOe1S38 


mo 206 


0.136 


02.150 


= Oreo 


0.584 


Oe 2 


Winner 2: None 


0 
0.0000 
0.0000 
0.0000 
0.0000 
310000 
0.0001 
0.0001 
op Relere) 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


PE: 


PE 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


waver: 


66 
1.000 
-0.436 
67 
1.000 
-0.086 
68 
1.000 
0.082 
69 
1.000 
-0.108 
70 
1.000 
Oo 7 1 
pel 
1.000 
O.182 
72 
1.000 
0.233 
We 
1.000 
-0.244 
74 
1.000 
O7-373 
TD 
1.000 
=O 5.6 
76 
1.000 
-0.484 
ia 
1.000 
0.128 
78 
1.000 
-0.047 
79 
1.000 
=0.379 
80 
1, OGO 
0.647 
Out 
PES: 


eae a 
Sum 


por 
Sum 


Err 
Sum 


BEE 
Sum 


Err 
Sum 


Eieds 
Sum 


og a 
Sum 


BrY 
Sum 


Jou abe 
Sum 


Ber 
Sum 


Er 
Sum 


Jayenge 
Sum 


Jage3e 
Sum 


Err 
Sum 


Err 
Sum 


al 


Spacing: 5 
Shape: Square 


Scale: 
Offset: 0.00 


Init Low: 


slits 


00 


=OL.LO0 
Winner 1: None 
L/R Schedule: out 


Recall Step 
Input Clamp 


Factor 0.000 Desired 
-0.410 Transfer 
Factor 0.000 Desired 
-0.086 Transfer 
Factor 0.000 Desired 
0.082 Transfer 
Factor 0.000 Desired 
-0.108 Transfer 
Factor 0.000 Desired 
0.071 Transfer 
Factor 0.000 Desired 
0.179 Transfer 
Factor 0.000 Desired 
0.229 Transfer 
Factor 0.000 Desired 
-0.239 Transfer 
Factor 0.000 Desired 
0.361 Transfer 
Factor 0.000 Desired 
-0.308 Transfer 
Factor 0.000 Desired 
-0.449 Transfer 
Factor 0.000 Desired 
0.127 Transfer 
Factor 0.000 Desired 
-0.047 Transfer 
Factor 0.000 Desired 
-0.361 Transfer 
Factor 0.000 Desired 
0.569 Transfer 
Wgt Fields: 2 
F' offset: 0.00 
Low Limit: -9999.00 
High Limit: 9999.00 


1 
0.0000 
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0.0000 


Init High: 0.100 


0 


0 
0.0000 


=6 


Or 


0 


.410 


. 086 


082 


- 108 


07m 


Li 


i, 


.239 


7 oO 


. 308 


449 


mle 


.047 


361 


969 


Sum: 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Sum 


Transfer: TanH 
Output: Direct 


Error Func: 


standard 


Learn: Delta-Rule 
L/R Schedule: out 
Winner 2: None 


0 


0.0000 


0 
0.0000 


—————— am ——SS lle 


Firing Density 100.0000 0.0000 0.0000 
Temperature 0.0000 0.0000 0.0000 
Gain 1.0000 0.0000 0.0000 
Gain 1.0000 0.0000 0.0000 
Learn Step 10000 30000 70000 
Coefficient 1 0.1500 0.0900 0.0324 
Coefficient 2 0.3000 0.1800 0.0648 
Coefficient 3 O-1006 0.1000 0.1000 
PE: 81 
1.000 Err Factor -0.525 Desired 
=O. 503 Sum -0.525 Transfer 
26 Weights O-0G0SErECr 


0.0000 
Q0.0000 
0.0000 
0.0000 
150000 
0.0042 
0.0084 
0. 1000 


0.0000 
0.0000 
0.0000 
0.0000 
310000 
0.0001 
0.0001 
0) SOLON, 


=Omec. Output 
02-000 Current Error 


KKK KKK KKK KKK KKK KKK KKK KKK RK KKKKRKKKRKRKEKKKKRKKKKRKKRKKRKKKKEKRKKEKEKKKKK 


Resulting actual output and desired output for encryption after 
convergence in accordance with Table 4.1 input: 


Desired: Actual: 

12828.000000 12827 .522461 
38939.000000 38939.464844 
41525.000000 41524 .664063 
16986.000000 16985 .642188 
25907 .000000 25907. 292969 
41305.000000 41304.957031 
34609.000000 34609.128906 


9880.000000 
37175 .000000 
26139 .000000 
4942.000000 
30022 .000000 
6523 .000000 


21386 .000000 © 


4780.000000 


9880.100586 
Be 1) 57384575 
26138 .814453 
4942.453223 
30021.833984 
S525. 165889 
21385 .605469 
4779.714844 


26946 .000000 26946 .346094 
33050 .000000 33050.152344 
29325 .000000 ZI 2242022200 
eat . 000000 137507502505 
62803 .000000 S2803e5 52031 
59988 .000000 59987 .847656 


3018.000000 

45353.000000 
54086 .000000 
50536.000000 
2445.000000 

54097 .000000 
45926 .000000 


S0L72 878906 

3 4 Oo 
594086 .285156 
50536 .437500 
2445 .414014 

54097 .246094 
45926.305469 


KKK KKK KKK KKK KKK KKK KKK KK KK KKK KK KKK KKK KE KKKKKKK KK KKK KKK 


Title: Decryption Network for In--Depth Example of Chapter 4 


Display Mode: Network 
Display Style: default 
Control Strategy: backprop 
333877 Learn 
16 Aux 1 
L/R Schedule: backprop 
Recall Step i 


O Recall 
Q Aux 2 


0 
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Type: Hetero-Associative 


L/R Schedule: backprop 


QO Layer 
QO Aux 3 


0 


Firing Density 100.0000 0.0000 0.0000 0.0000 0.0000 
Gain 1.0000 0.0000 0.0000 ~°0.0000 0.0000 
Learn Step 5000 0 0 0 0 
Coefficient 1 0.9000 0.0000 0.0000 0.0000 0.0000 
Coefficient 2 0.6000 0.0000 0.0000 0.0000 0.0000 
Coefficient 3 0.0000 0.0000 0.0000 0.0000 Q.Q000 
IO Parameters 
Learn Data: File Rand. (decryption file) Binary 
Recall Data: File Seq. (decryption) 
Result File: Desired Output, Output 
UserIO Program: userio 
I/P Ranges: =LOgOU, 1.0000 
O/P Ranges: -0.8000, 0.8000 
I/P Start Col: i MinMax Table: samb 
O/P Start Col: 5 Number of Entries: 5 
MinMax Table <samb>: 
Col: it 2 3 4 5 
Min: 0.0000 1, 000¢ 1.0000 1.0000 2445 .0000 
Max: 15 ial ly, 14 6. 28e+004 
Layer: 1 
PEs: 1 Wgt Fields: 2 Sum: Sum 
Spacing: 5 F' offset: 0.00 Transfer: Linear 
Shape: Square Output: Direct 
Scale: 1.00 Low Limit:329959 2200 Error Func: standard 
Offset: 0.00 High Limit: 9999.00 Learn: --None-- 
Init Low: -0.100 Init High: s0eeo L/R Schedule: (Network) 
Winner 1: None Winner 2: None 
PE: Bias 
1.000 Err Factor 0.000 Desired 
0.000 Sum 1.000 Transfer 1.000 Output 
0 Weights —-247 .657 eh Ener 0.000 Current Error 
Layer: In 
PEs: 4 Wgt Fields: 1 Sum: Sum 
Spacing. =) F' offset: 0.00 Transfer: Linear 
Shape: Square Output: Direct 
Scale: 1.00 Low Limit: -9999.00 Error Func: standard 
Offset: 0.00 High Limit: 9999.00 Learn: --None-- 
Init Low: -0.100 Init High: ©7100 L/R Schedule: (Network) 
Winner 1: None Winner 2: None 
PE: 2 
1.000 Err Factor 0.333 Desired 
0.333 Sum 0.333 Transfer 0.333 Output 


x**x Q Weights 0.000 Error 0.000 Current Error 
xxx Repeat for PE's here on, 0 weights, O error. 


PE: 3 
1000 Erisraceer -1.000 Desired 
-1.000 Sum -1.000 Transfer -1.000 OQutpug 
PE: 4 
1.000 Err Facter -0.273 Desired 
S00 2/3. Sn -0.273 Transfer -0.273 Output 
PE: 5 
1.000 Err Factor 0.231 Desired 
0.231 Sum 0.231 Transfer 0.231 OuGpua 
Layer: Hiddenl 
PEs; 25 Wgt Fields: 2 Sum: Sum 
Spacing > F' offset: 0.00 Transfer: TanH 


88 


Offset: 0.00 
-0.100 


Shape: Square 
Scale: 1.00 


Init Low: 


Winner 1: None 


Low 


L/R Schedule: hiddenl 


Limit: 


=9999 .00 
Highwirmit: 9999.00 
IGehemestejag i. alee 


Error Func: 


OUEDUT: 


Direct 


standard 
Learn: Delta-Rule 


L/R Schedule: hiddenl 


Winner 2: None 


Recall Step al 0 0 0 0 
Firing Density 100.0000 0.0000 0.0000 0.0000 0.0000 
Gain 1.0000 0.0000 0.0000 0.0000 0.0000 
Gain 1.0000 0.0000 0.0000 0.0000 0.0000 

Learn Step 10000 30000 70000 150000 310000 
Coefficient 1 0.3000 0.1500 0.0375 0.0023 0.0000 
Coefficient 2 0.3000 0.1500 ©. 0:37 5 0.0023 0.0000 
Coefficient 3 0.1000 0.1000 (oes Rolere 0.1000 0.1000 

PE: 6 

mOoO0 Err Factor 0.000 Desired 
1.734 Sum 0.940 Transfer 0.940 Output 
xxx 5 Weights =O, 000" Error -Q.000 Current Error 
**x Repeat for PE's from here on, 5 weights, nearly 0 error. 
PE: 7 
7000 Err Factor 0.000 Desired 
aeelil Sum -0.971 Transfer Oo PeOut put 
PE: 8 
m000 Err Factor 0.000 Desired 
mee297 Sum -0.289 Transfer sO zoo Output 
PE: 9 
meoO0 Err Factor 0.000 Desired 
0.912 Sum 0.722 Transfer OF 2 22Cut put 
PE: 10 
m000 Err Factor 0.000 Desired 
Zoezoo Sum B0ne 2 cetrans fer SOc. CULDUT 
eee 11 
mwO0) Err Factor 0.000 Desired 
=O. 159 Sum -0.158 Transfer aOeeL SS GOucDUt 
PE: 12 
ioO0 Err Factor 0.000 Desired 
0.169 Sum 0.168 Transfer 0.168 Output 
PE: 13 
1.000 Err Factor 0.000 Desired 
-0.342 Sum -0.330 Transfer = ORs oO SOUT DUT 
PE: 14 ; 
1.000 Err Factor 0.000 Desired 
0.677 Sum 0.589 Transfer 0.589 Output 
PE: 15 
1.000 Err Factor 0.000 Desired 
-1.055 Sum -0.784 Transfer 05/84 Output 
PE: 16 
ioog Err Factor 0.000 Desired 
Orzo SUM -Q.212 Transfer = Omir. OuGpUE 
BE: 17 
imoOO@Err Factor 0.000 Desired 
1.487 Sum 0.903 Transfer Oe 903, 0utput 
PE: 18 
1.000 Err Factor 0.000 Desired 
BOs 220 Sum -0.245 Transfer -0.245 Output 
PE: 19 
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12 00G" Exeebactor 0.000 Desired 
0.158 Sum 0.156 Transfer 
PE: 20 
1.000 Err Factor 0.000 Desired 
1.666 Sum 0.931 Transfer 
PE: 21 
1. 000-Err Factor 0.000 Desired 
-2.920 Sum -0.994 Transfer 
PE: 22 
1.000 Err Factor 0.000 Desired 
0.136 Sum 0.135 Transfer 
PE: 23 
1.000 Err Factor 0.000 Desired 
0.118 Sum 0.117 Transfer 
PE: 24 
1.000 Err Factor 0.000 Desired 
-0.597 Sum -~0.535 Transfer 
PE: 25 
1.000 Err Factor 0.000 Desired 
0.154 Sum 0.153 Transfer 
PE: 26 
1.000 Err Factor 0.000 Desired 
0.203 Sum 0.201 Transfer 
PE: 27 
1.000 Err Factor 0.000 Desired 
—ieo50 Sul -0.876 Transfer 
PE: 28 
MEOOO Err Factor 0.000 Desired 
0.508 Sum 0.468 Transfer 
PE: 29 . 
1.000 Err Factor 0.000 Desired 
=. 387 Sum -0.955 Transfer 
PE: 30 
1.000 Err Factor 0.000 Desired 
0.345 Sum 0.332 Transfer 
Layer: Hidden2 
PEs: 25 Wgt Fields: 2 
Spacing: 5 F' offset: 0.00 
Shape: Square 
Scale: 1.00 Low Limit: -9999.00 
Offset: 0.00 High Limit: 9999.00 
Init Low: -0.100 Init High: 0.100 


Winner 1: None 


L/R Schedule: hidden2 
Recall Step 
Firing Density 100.0000 

Gain 

Gain 
Learn Step 


Coefficient 1 
Coefficient 2 
Coefficient 3 


PE: 


keR* 


Si 


Error Func: 


0.156 


0 .. 232) 


994 


0. 133 


0. De 


36 


0.138 


0. 20m 


. 876 


0.468 


=02 


215 5 


0.1332 


Sum: 


Transfer: 


Output: 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Sum 


TanH 
Direct 


standard 
Learn: Delta-Rule 


L/R Schedule: hidden2 


1 Q Q 
0.0000 0.0000 

1.0000 0.0000 0.0000 
1.0000 G20000 0.0000 
10000 30000 70000 
022506 G2 1250 G20513 
O230C0 O21500 Os OauS 
Oe LOC? 0.1000 Gg. 1060 


1,000 Err Factor 


-4.909 


26 Weights 


Sum 


0.000 Desired 


-~1.000 Transfer 
—0  COGMERror 


90 


Q 
0.0000 
0.0000 
0.0000 
150000 
0/0020 
O70022 
O07 LeoG 


Winner 2: None 


Q 
0.0000 
0.0000 
0.0000 
310000 
0.0000 
0.0000 
OZ ocr 


~1 . 000,0UuErpuE 
~0.000 Current Error 


**xk Repeat for PE's here on, 
PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


SZ 
f 000 
mr. OSS 
33 
tO 00 
3.423 
34 
1.000 
3 SEE, 
BD 
1.000 
0.414 
36 
1.000 
mir 2) SD 
3) 
1.000 
2 O 
35 
1.000 
B08 / 
39 
1.000 
es | 
40 
1.000 
gO.3/9 
41 
O00 
0.636 
42 
1.000 
nO.023 
43 
1.000 
C019 
44 
= O00 
pe. 200 
45 
1.000 
2516 
46 
000 
i206 
47 
0 00 
Ome) 2 
48 
1.000 
1.743 
49 
1.000 
= Lao 
50 


Jee 
Sum 


pele 
Sum 


| a eg 
Sum 


Err 
Sum 


| dane 
Sum 


Err 
Sum 


ice 
Sum 


BrEr 
Sum 


ldygic 
Sum 


Err 
Sum 


ECr 
Sum 


ese 
Sum 


Err 
Sum 


lagen 
Sum 


jy aa a 
Sum 


oe 
Sum 


dp ena 
Sum 


Ear 
Sum 


Pace Or 


Factor 


Paccor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Paceor 


Factor 


Factor 


Pacctor 


Faceern 


Pactcor 


Paccor 


Pactor 


Factor 


Oo OO oe) Oo Oo OO OO Oo OO OO OO OO OO O O 


26 weights, nearly O error. 


Oo: 
ij 2p 


0 


OO 


OO 


O © 


000 


. 000 
soos 


.000 
«298 


. 000 
ao 


.000 
eB 


.000 
.949 


.000 
-J99 


.000 
854 


.000 
43.62 


.000 
503 


. 000 
Poy / 


. 000 
~550 


000 
OS 


000 
Poof 


. 000 
. 836 


-000 
.750 


000 
~941 


000 
2206 
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Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


O79 


O96 


O96 


O22 


Ono 55 


0.949 


0.799 


0.854 


023602 


Og 563 


=O. 677 


0. 550 


=O. 90> 


O26 7 


O77 636 


0.750 


0.941 


—07.. 908 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Output 


Cuepuc 


Output 


Output 


CuEpuE 


Output 


Output 


Output 


Output 


OuEpU Ee 


Output 


1.000 Err Factor 0.000 Desired 
Oslos= sum 0.165 Transfer 0.165 Output 
PE: 51 
1.000 Err Factor 0.000 Desired 
0.276, Sum 0.264 Transfer 0.264 Output 
PEsmoe 
1.000 Err Factor 0.000 Desired 
OF LZ) soum 0.124 Transfer 0.124 Output 
PE: 53 
IPSCO, i dpese VHsle yoga 0.000 Desired 
—l.3365S5um -0.871 Transfer -0.8715 Ouepue 
PE: 54 
1.000 Err Factor 0.000 Desired 
0% 956 soul -0.744 Transfer -0.7443OUeeuG 
PE: 55 
1.000 Err Factor 0.000 Desired 
O25s34.Sum 0.488 Transfer 0.488 Output 
Layer: Hidden3 
PEs: 25 Wgt Fields: 2 Sum: Sum 
Spacing: 5 F' offset: 0.00 Transfer: TanH 
Shape: Square Output: Direct 
Scale: 1.00 Low Limit: -9999.00 Error Func: standard 
Offset: 0.00 High Limit: 9999.00 Learn: Delta-Rule 
Init Low: -0.100 Init High: 0.100 L/R Schedule: hidden3 


Winner 1: None Winner 2: None 


L/R Schedule: hidden3 


Recall Step a 0 0 0 0 
Firing Density 100.0000 0.0000 0.0000 0.0000 0.0000 
Gain 1.0000 0.0000 0.0000 0.0000 0.0000 
Gain LOCO 0.0000 0.0000 0.0000 0.0000 

Learn Step 10000 30000 70000 150000 310000 
Coefficient 1 0.2000 OFLOOO 020250 O-001G 0.0000 
Coefficient 2 0.3000 0.1500 O20375 O02 0.0000 
Coefficient 3 0.1000 0.1000 0.1000 0.1000 0.1000 

PE: 56 

LaG00, Err Factor 0.000 Desired 
0.824 Sum 0.677 Transfer 0.6/7 Gtitput 
xxx 26 Weights =0), COO sError -0.000 Current brace: 
xxx Repeat for PE's here on, 26 weights, nearly O error. 
PE >/ 
1 CUGVErr  ractenr 0.000 Desired 
O2a25 sum 0.317 Transfer 0.317 Gumpue 
PEL 58 
i000. Err Factor 0.000 Desired 
[Ons s Sum -0.131 Transfer -Q.131 OUESE 
PE: 59 
1.000 Err Factor 0.000 Desired 
[0-03 5) Sum —0). Os>alwanster -0.035 OQutpsue 
PE: 60 
1.000 Err Factor 0.000 Desired 
=0 31207 Sum -0.120 Transfer -0.120 Outage 
PE 6 : 
1.000 Err Factor 0.000 Desired 
-0.671 Sum -0.586 Transfer -0.586 Output 
PE: 62 
TetOOC thers accor 0.000 Desired 


az 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


PE: 


Layer: 


BO. 11.0 
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TUR OT AS: 
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1.000 
Ono 9] 
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ir O00 
BO.Ll? 
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m2: 009 
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1.000 
mu. 735 


i O00 
mo. L42 


1.000 
0.405 
f2 

1.000 
o. 007 
3 

1.000 
Bros 
74 

1.000 
GE238 


ILC 16 Ke, 
-0.478 


i OGo 
mo. 208 


i 000 
0.474 


1.000 
-8.096 
79 
1.000 
0.169 
80 
1.000 
=Ouzo | 
Out 


Sum 


Maa 
Sum 


Err 
Sum 


Err 
Sum 


jal e 
Sum 


I a ay 
Sum 


Err 
Sum 


Err 
Sum 


a a 


Sum 


je 
Sum 


Err 
Sum 


Err 
Sum 


Ber 
Sum 


Err 
Sum 


Err 
Sum 


Sige 
Sum 


Err 
Sum 


Err 
Sum 


Ber 
Sum 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


baGeor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


L1L0 


. 000 
076 


. 000 
sOuZ 


. 000 
1085 


.000 
ol? 


.000 
5 ed oks 


. 000 
~472 


.000 
70126 


. 000 
~141 


.000 
. 384 


.000 
007 


. 000 
Bee pe)s) 


.000 
~ 234 


.000 
~444 


~000 


5 PLEO, 


. 000 
-441 


. 000 
.000 


.000 
5 Aloe) 


. 000 
25 
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Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


Desired 
Transfer 


FLO 


076 


.602 


poe 


rey 


. 968 


.472 


626 


~141 


384 


SOO7 


2999 


- 234 


-444 


. 280 


~441 


. 000 


- 6; / 


now 


Output 
Cucoue 
Ctejebhe 
Output 
Out pute 
Sutrput 
QOuEpUE 
CuEPUT 
Output 
Output 
Output 
Output 
Output 
Output | 
Output 
Output 
Oulepu 
Output 


Ouie pu 


Scale: 


PEs: 1 
Spacing = 
Shape: Square 


1.09 


Offset: 0.00 


Init Low: 


=0. 200 


Wgt Fields: 2 
' of fSetnmOneo 


F 


Low Limit: 


=9999. 00 


High Limit: 9999.00 
Init High: 0.100 


Winner 1: None 
L/R Schedule: out 


Recall Step 


1 


Firing Density 100.0000 
Gain 
Gain 
Learn Step 
Coefficient 1 
Coefficient 2 
Coefficient 3 


PE: 


el 


LIOOG. Err 
-0.307 Sum 
26 Weights 


Factor 


1.0000 
120000 

10000 
0.1500 
G2 3000 
0.1000 


0 
0.0000 
0.0000 
0.0000 

30000 
O207.50 
Bek Sele 
0.1000 


Sum: Sum 
Transfer: TanH 
Output: Direct 
Error Func: standard 
Learn: Delta-Rule 
L/R Schedule: out 


Winner 2: None 


-0.298 Desired 


-0.298 Transfer 


0.000 Error 


@) 0 @) 
0.0000 0.0000 0.0000 
0.0000 0.0000 0.0000 
0.0000 0.0000 0.0000 

70000 150000 310000 
0.0188 O.0012 0.0000 
O-70379 5 0.0023 8) Toke 9), 
GC pale ene 0.1000 0.1000 

-0.298 Olieput 
0.000 Current Error 
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Decryption desired and actual output after convergence 
according to input of Table 4.1: 


Desired: 

4780.000000 
4942.000000 
65732 000000 
9880.000000 
37 542 
.000000 
26946. 
26139. 
29375. 
BOO22. 
-000000 
- 000000 
37175. 
38939. 
.000000 
41525. 
-000000 
12828. 


21386 


S2050 
34609 


41305 
25907 
fog936 
45926 
45353 


50530 
54086 


a2e 03 


000000 
000000 
000000 


Q00000 
000000 


Q00000 
Q00000 
000000 


000000 


. 000000 
-000000 
. 000000 
. 000000 
- 000000 
54097. 
Dogo 


000000 
000000 


- 000000 
3018 .000000 
2445 .000000 


Actual: 


4179.94 9310 
4941.904785 
8523.464258 
Jeu, 2aoce 


13750. 
21365: 
26945. 
201367 
e232 
SUOZ2. 
33049. 
34609. 
Saas 
389390 
41305. 
41525. 
25707. 
tZe20- 
To9goe 
45925. 
nono] ote 
= 0/5)S\5)- 
54086. 
54097. 
59988. 
6200s 


194336 
947266 
638672 
SOLIS 3 
D0 50S 
140625 
Zones 
441406 
546875 
PESNPLEN G2, 
357031 
300781 
408984 
163086 
839844 
791406 
366406 
578906 
265625 
Zoos ot 
027344 
003906 


5017 30 / Gal 
2444 .980957 
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